Editing media using graphical representation of media

ABSTRACT

A non-transitory machine readable medium that has a computer program for adjusting color values of an image represented in a color space is described. The computer program displays a graphical representation of the color values of the image. The computer program receives a set of inputs on the graphical representation. In response to the received set of inputs, the computer program determines a color space transform for mapping a set of color values defined within the color space to a different set of color values. The computer program modifies the image by using the color space transform to change the color values of the image.

CLAIM OF BENEFIT TO PRIOR APPLICATION

This application claims benefit to U.S. Provisional Patent Application61/595,650, entitled “Intuitive Media Editing,” filed Feb. 6, 2012. TheU.S. Provisional Patent Application 61/595,650 is incorporated herein byreference.

BACKGROUND

Digital graphic design, image editing, and video editing applications(hereafter collectively referred to as media content editingapplications or media editing applications) provide graphical designers,media artists, and other users with the necessary tools to view,analyze, and edit a variety of media content. Examples of suchapplications include iPhoto®, Aperture®, Final Cut Pro® and iMovie®, allsold by Apple, Inc. These applications give users the ability to analyzeand edit images and videos in a variety of manners. For example, someapplications provide different graphical representations of the colorvalues of an image (e.g., histogram representation, waveformrepresentation, vectorscope representation, etc.) to show users thedistribution of tones (luminance levels) or distribution of colors in animage. In addition, some applications provide a user interface thatincludes adjustment sliders and other tools for editing the image.

Many media-editing applications, however, do not provide intuitive coloradjustment controls. For example, the user is required to have extensiveknowledge about color editing in order to effectively use most of theexisting color adjustment tools. Furthermore, the controls for adjustingdifferent aspects of the color values of an image are dispersed indifferent locations of the user interface. These deficiencies causeunnecessary inconvenience in editing an image.

BRIEF SUMMARY

Some embodiments of the invention provide several novel user interface(UI) tools for editing a set of images in a media-editing application.For instance, the media-editing application of some embodiments displaysa graphical representation of a set of color values of an image andallows a user to edit the color values of the image by modifying thegraphical representation. In this manner, the graphical representationserves as a UI tool that the user can select and manipulate. Based onthe user's manipulation of the graphical representation, the applicationgenerates a color space transform. The color space transform maps colorsdefined within a color space in which the color values of the image aredefined to a different set of colors.

After generating this color space transform, the application uses thetransform to adjust the color values of the image. In some embodiments,the image that is currently being edited is one of several pictureframes of a video clip. In some of these embodiments, the applicationuses the generated color space transform to adjust color values of oneor more of the other picture frames.

A graphical representation of the color values of an image presents oneor more aspects of the color values in a graphical form. Differentgraphical representations present the image's color values in differentmanners. For example, a color channel histogram shows a distribution ofcolor channel values of different pixels across the image. A waveformscope shows a distribution of luminance values (e.g., brightness values)of different pixels at each horizontal location of the image. Avectorscope displays a distribution of chrominance values (e.g., colorshades and color intensity) of different pixels in the image.

The media-editing applications of different embodiments use differenttechniques to receive user input through the displayed, color-valuegraphical representations. For instance, the application of someembodiments identifies the location and movement of the user input(e.g., the cursor or touch screen input), and correlate this locationand movement to a location and a vector within the graphicalrepresentation. More specifically, a user can select a location on thegraph by placing a cursor on the graph and providing an input (e.g.,clicking a button of a cursor controlling device, pressing a hot key,etc. or by touching an area of a touch screen display that displays thegraph). In these embodiments, the media-editing application determines aposition of the user input (the cursor position or the position of thetouch) and corresponds that position to a location on the graph. Theposition of the user input can be determined by identifying a set ofcoordinates (e.g., a horizontal coordinate x and a vertical coordinatey) of the user input. In addition, the user can provide a vector bymoving the cursor (or moving the finger on the touch screen device) to adifferent position.

In other embodiments, the media-editing application provides severalselectable UI items that are located at different sections on the graph.The user may select a point on the graph by selecting one of theselectable UI items. In addition, the user may provide a vector on thegraph by moving the selected UI item to a different location on thegraph.

After receiving the set of user inputs, the media-editing application ofsome embodiments uses the selected location on the graph and the vectorinput to identify a particular color value adjustment. Specifically, theapplication first identifies a particular color value that correspondsto the selected location on the graph (e.g., a luminance value thatcorresponds to the selected location on a waveform representation). Insome embodiments, when the graphical representation presents the colorvalues in more than one dimension (e.g., a vectorscope representationthat represents the color shade and color intensity of the colorvalues), the application identifies a particular set of color valuesthat correspond to the selected location of the graph (e.g., a set ofchrominance values that correspond to the selected location on avectorscope representation). The application then uses the vector inputto determine an adjustment to the particular color value. In someembodiments, the application uses the direction of the vector input todetermine an increase or a decrease of the particular color value. Forexample, a first vector direction may correspond to an increase of theparticular color value while a second vector direction may correspond toa decrease of the particular color value. In addition, the applicationuses the magnitude of the vector to determine an amount of adjustmentthat is applied to the identified color value. For example, a largervector magnitude corresponds to a larger color value adjustment and asmaller vector magnitude corresponds to a smaller color valueadjustment.

After determining the particular color value adjustment, the applicationof some embodiments determines amounts of adjustment for other colorvalues (e.g., adjustments for other luminance values) that arerepresented in the graphical representation. Different techniques areused in different embodiments to determine adjustments to the othercolor values. For instance, the application of some embodimentsconstructs a blended Gaussian curve to determine the amounts ofadjustments that are applied to the different color values. A Gaussiancurve is a bell shaped curve that is defined along two axes, ahorizontal axis that specifies values along one component colordimension (e.g., luminance) and a vertical axis that specifies differentamounts of adjustments. In some embodiments, the application constructsthe Gaussian curve by first identifying a horizontal location on theGaussian curve that corresponds to the particular color value and thencreating a bump (i.e., a bell shaped curve) that has a peak at thathorizontal location. The height of the peak is determined by the amountof adjustment for the particular color value. The application may alsoblend this newly constructed Gaussian curve with any existing Gaussiancurves that were constructed based on previously provided input tocreate a blended Gaussian curve. By using this blended Gaussian curve,the application identifies different amounts of adjustment to be appliedto different color values in the image.

Instead of constructing Gaussian curves, the media-editing applicationof some embodiments creates a falloff region on the graphicalrepresentation in order to determine adjustments to the other colorvalues. The falloff region is a region on the graphical representationwith the center of the region located at the user's selected location.In these embodiments, the application gradually reduces the amount ofadjustments as the representations of the color values are locatedfarther away from the center of the falloff region. In addition, noadjustment will be applied to color values located outside of thefalloff region.

In addition, the application of some embodiments applies a fluid modelwhen determining adjustments to the different color values. Under thefluid model, the application first divides the vector input into severalsmaller vectors and determines adjustments to the color values as if thesmaller vectors are provided on the graphical representation. This fluidmodel allows the graphical representation to shift and/or skew smoothlyto create a smooth adjustment to the image and minimize the effect ofposterization.

Based on the determined adjustments to the different color values, themedia-editing application generates a color space transform that mapscolors defined within the color space to a different set of colors. Insome embodiments, the color space transform is an algorithm or amathematical formula that takes an input color that is defined withinthe color space and generates an output color. Before applying the colorspace transform to an image, the media-editing application of someembodiments uses the color space transform to construct a lookup table.The lookup table includes a set of input colors defined within the colorspace and a set of corresponding output colors that are generatedaccording to the color space transform. After the lookup table isconstructed, the application adjusts the colors of the image byreplacing each color in the image with the color's corresponding outputcolor based on the lookup table. Instead of constructing a lookup table,the media-editing application of some other embodiments use thealgorithm or mathematical formula directly on the different color valuesof the image to change the colors of the image in real-time.

After applying the color space transform to the image, the applicationupdates the graphical representation of the image according to theadjustments made to the color values. In some embodiments, the colorspace transform causes the graphical representation to shift and/or skewin a manner that is consistent with the user's input on the graphicalrepresentation. Specifically, the graphical representation shifts and/orskews in the same direction as the user's vector input. In addition, thedegree to which the graphical representation shifts and/or skews isproportional to the magnitude of the user's vector input.

Some embodiments provide another novel user interface for editing a setof images in a media-editing application. In these embodiments, theapplication provides a color selection tool that is integrated with thecolor channel histogram representation of the color values of an image.

The color selection tool allows a user to select a specific color forediting the set of images. Different embodiments implement the colorselection tool differently. In some embodiments, the color selectiontool is a color pot that displays different colors of the color space.In other embodiments, the color selection tool displays an input fieldon the GUI of the application for a user to enter a specific set ofcolor values.

In some embodiments, the color selection tool does not always appear onthe color channel histogram representation of the image, but only do sowhen the color selection tool is activated. The color selection tool maybe activated in different manners. In some embodiments, the user mayactivate the color selection tool by selecting a location on thehistogram. The user may select a location on the histogram in differentmanners as described above. In these embodiments, the applicationidentifies a particular color value that corresponds to the selectedlocation on the histogram.

Once the user selects a color using the color selection tool, theapplication of some embodiments generates a color space transform forthe image. In some embodiments, the color space transform is generatedbased on the selected color and the particular color value thatcorresponds to the selected location on the histogram. Specifically, thecolor space transform changes colors within the color space to be moresimilar to the selected color. In addition, the changes that are appliedto different colors are not constant. For example, a larger amount ofchanges is applied to colors having color values closer to theparticular color value, and a smaller amount of changes is applied tocolors having color values farther away from the particular color value.After generating this color space transform, the application uses thetransform to adjust the color values of the image.

As another novel user interface, some embodiments of the inventionprovide a color channel histogram that includes a range beyond thepredetermined black and white luminance cutoff values of the image. Theblack luminance cutoff value identifies a luminance value as black andthe white luminance cutoff value identifies a luminance value as whitewhen the image is displayed or reproduced.

Pixels of an image may contain luminance values that are beyond therange of black and white luminance cutoff values under differentscenarios. For example, modern image-capturing devices may capture andstore color values beyond the black and white luminance cutoff values inan image. Certain image file formats, such as formats that supporthigh-dynamic range, may support color values that are outside the rangeof the black and white luminance cutoff values. Also, luminance valuesof some pixels that are originally within the range of the black andwhite luminance cutoff values may be pushed beyond the range by a userwho is editing the image (e.g., increasing or decreasing the brightnesslevel of at least some pixels of the image). Thus, the media-editingapplication of some embodiments provides a color channel histogramrepresentation that includes a range beyond the predetermined black andwhite luminance cutoff values. Consequently, all pixels of the image,including pixels having luminance values darker than the black luminancecutoff value and pixels having luminance values brighter than the whiteluminance cutoff value, are accurately presented in the histogram.

By accurately presenting pixel data in a color channel histogramrepresentation, a user may properly analyze and edit the image to adesired effect. For example, after learning that some of the pixels haveluminance values that are brighter than the white luminance cutoff value(displayed as pure white) or darker than the black luminance cutoffvalue (displayed as pure black), a user may edit the image by bringingthe luminance values of those pixels back within the range of black andwhite luminance cutoff values. In some embodiments, the user may performsuch an edit by directly manipulating the color channel histogramrepresentation of the color values of the image.

In addition to allowing a user to edit an image by directly manipulatingthe graphical representations of the image, the application of someembodiments displays a set of color adjustment tools on the image andallow a user to edit the color values of the image by controlling thecolor adjustment tools that are displayed on the image. In someembodiments, the color adjustment tools do not always appear on theimage, but only do so when the color adjustment tools are activated.

In some embodiments, the application displays a color adjustment toolthat simulates a graphical representation of the color values of theimage. In these embodiments, the user may provide an additional input(e.g., a vector) on the displayed image as if the user is manipulatingthe actual graphical representation. The application then generates acolor space transform based on the color values of the particular pixeland the vector input and adjusts the color values of the image in asimilar manner as described above. The application also updates theactual graphical representation of the color values of the imageaccording to the adjustments to the image's color values. In someembodiments, the effects on the image and the graphical representationare the same and it seems as if the user provides the inputs directly onthe graphical representation.

Instead of or in addition to displaying a color adjustment tool thatsimulates a graphical representation, some embodiments displays a set ofrange sliders for adjusting color values of different color channels.For example, the set of range sliders may include a range slider foradjusting luminance values, a range slider for adjusting red colorchannel values, a range slider for adjusting green color channel values,and a range slider for adjusting blue color channel values. Similar tothe color adjustment tools that simulate a graphical representation, theapplication generates a color space transform for the image based on thecolor values of the particular pixel and the inputs received on therange sliders. After generating the transform, the application uses thecolor space transform to adjust the color values of the image.

In some embodiments, the color adjustment tools are overlaid on thedisplayed image around the selected point on the image. In addition, thecolor adjustment tools are translucent such that they do not obstructthe user from viewing the displayed image.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description and the Drawings is needed.Moreover, the claimed subject matters are not to be limited by theillustrative details in the Summary, Detailed Description and theDrawings, but rather are to be defined by the appended claims, becausethe claimed subject matters can be embodied in other specific formswithout departing from the spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates an example of editing an image when a user directlymanipulates the graphical representation of the color values of theimage.

FIG. 2 conceptually illustrates a process of some embodiments forediting an image when a user directly manipulates the graphicalrepresentation of the color values of the image.

FIG. 3 conceptually illustrates a software architecture of amedia-editing application of some embodiments and an example color spacetransform resulted from manipulation of a graphical representation.

FIG. 4 illustrates an example of editing an image when a user providesinputs directly on the displayed image.

FIG. 5 illustrates an example of editing an image when a user directlymanipulates a waveform representation of the color values of the image.

FIG. 6 illustrates two example user interface implementations forallowing a user to manipulate a waveform representation of the colorvalues of an image.

FIG. 7 conceptually illustrates a software architecture of amedia-editing application of some embodiments and an example color spacetransform resulted from manipulation of a waveform representation.

FIG. 8 illustrates an example of generating a Gaussian curve based on amanipulation of a waveform representation of the color values of animage.

FIG. 9 illustrates an example of a blended Gaussian curve.

FIG. 10 conceptually illustrates a process of some embodiments forediting an image when a user directly manipulates the waveformrepresentation of the color values of the image.

FIG. 11 illustrates an example of editing an image when a user directlymanipulates a vectorscope representation of the color values of theimage.

FIG. 12 illustrates two example user interface implementations forallowing a user to manipulate a vectorscope representation of the colorvalues of an image.

FIG. 13 conceptually illustrates a software architecture of amedia-editing application of some embodiments and an example color spacetransform resulted from manipulation of a vectorscope representation.

FIG. 14 illustrates two examples of falloff regions created based on avector input on a vectorscope representation of an image's color values

FIG. 15 conceptually illustrates a process of some embodiments forcreating step vectors from a vector input on a vectorscoperepresentation of an image's color values.

FIG. 16 conceptually illustrates a process of some embodiments forediting an image when a user directly manipulates the vectorscoperepresentation of the color values of the image.

FIG. 17 illustrates an example of editing an image when a user directlymanipulates a color channel histogram representation of the color valuesof the image.

FIG. 18 illustrates two example user interface implementations forallowing a user to manipulate a color channel histogram representationof the color values of an image.

FIG. 19 conceptually illustrates a software architecture of amedia-editing application of some embodiments and an example color spacetransform resulted from manipulation of a color channel histogramrepresentation.

FIG. 20 illustrates an example of generating a Gaussian curve based on amanipulation of a color channel histogram representation of the colorvalues of an image.

FIG. 21 conceptually illustrates a process of some embodiments forediting an image when a user directly manipulates the color channelhistogram representation of the color values of the image.

FIG. 22 illustrates two example color space transforms resulted fromidentical inputs provided on the color channel histogram representationsof the color values of two different images.

FIG. 23 illustrates an example of editing an image when a user providestwo manipulations on a color channel histogram representation of thecolor values of the image.

FIG. 24 illustrates an example of editing an image when a user selects acolor from a color pot on a color channel histogram representation ofthe color values of the image.

FIG. 25 conceptually illustrates a process of some embodiments forediting an image when a user selects a color from a color pot on a colorchannel histogram representation of the color values of the image.

FIG. 26 illustrates an example of providing a broader range of luminancevalues on a color channel histogram representation of the color valuesof the image.

FIG. 27 illustrates an example of manipulating a color channel histogramrepresentation that has a broader range of luminance values.

FIG. 28 illustrates a different example of manipulating a color channelhistogram representation that has a broader range of luminance values.

FIG. 29 illustrates an example of editing an image by providing inputsdirectly on the displayed image.

FIG. 30 illustrates another example of editing an image by providinginputs directly on the displayed image.

FIG. 31 illustrates an example user interface implementation forallowing a user to edit an image by providing inputs directly on thedisplayed image.

FIG. 32 conceptually illustrates a process of some embodiments forediting an image when a user provides inputs directly on the displayedimage.

FIG. 33 conceptually illustrates the software architecture of a videodecoder.

FIG. 34 conceptually illustrates an electronic system 3400 with whichsome embodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

Some embodiments of the invention provide several novel user interfaces(UIs) for editing a set of images in a media-editing application. Forinstance, the media-editing application of some embodiments provides agraphical representation of the color values of an image and allows auser to edit the color values of the image by modifying the graphicalrepresentation. In this manner, the graphical representation serves as atool in the user interface that the user can select and manipulate.Based on the user's manipulation of the graphical representation, theapplication generates a color space transform. The color space transformmaps colors defined within a color space in which the color values ofthe image are defined to a different set of colors. After generatingthis color space transform, the application uses the transform to adjustthe color values of the image.

In some embodiments, the media-editing application is an applicationthat enables a user to edit one or more images. Through a graphical userinterface (GUI) of the application, the user specifies the manner inwhich the images are edited. In some embodiments, the user edits animage by modifying the color values of at least one pixel of the image.

FIG. 1 illustrates an example GUI 100 of a media-editing application ofsome embodiments that allows a user to edit an image by directlymanipulating a graphical representation of the color values of the imageat four different stages 105, 110, 115, and 120. Each of these stageswill be described in more detail below.

As shown in FIG. 1, the GUI 100 illustrates an image 125 and a graphicalrepresentation 140 of the color values of the image 125. The image 125is a picture of a person canoeing under a sunny sky. As mentioned above,a graphical representation presents color data of the image in agraphical manner. In this example, the graphical representation 140shows a distribution of luminance values of the different pixels acrossthe image 125. The horizontal axis of the graph 140 corresponds todifferent luminance values. The origin 135 of the horizontal axis of thegraph 140 corresponds to a minimum luminance value and a point 145 ofthe horizontal axis corresponds to a maximum luminance value within thecolor space. Pixels of the image 125 that have higher luminance values(i.e., pixels that are brighter) are represented in the right portion ofthe graphical representation 140 while pixels of the image 125 that havelower luminance values (i.e., pixels that are darker) are represented inthe left portion of the graphical representation 140.

The media editing operation will now be described by reference to thestate of the GUI 100 during the four stages 105, 110, 115, and 120. Thefirst stage 105 illustrates the GUI 100 of the media-editing applicationwhen the image 125 has just been opened. As shown, the image 125 is verydark, as illustrated by the diagonal lines across the image 125. Thegraphical representation 140 also shows a concentration of pixels in theleft portion of the graph 140, indicating that most pixels of the image125 have low luminance values.

The second stage 110 illustrates the GUI 100 after the user hasinitiated the media editing operation by selecting a location 150 on thegraph 140. The selection of the location on the graph 140 may beperformed by placing a cursor at a desired location on the graph 140 andproviding an input (e.g., clicking on the cursor controlling device,pressing a hot key, etc. or touching an area on a touch screen devicethat displays the graph 140). Although the user in this example hasselected a location on the graph 140 that overlaps with an area thatrepresents the color values of the image 125, the user may also select alocation on the graph 140 that does not overlap with such an area inorder to initiate the media editing operation.

The third stage 115 illustrates the GUI 100 after the user beganmanipulating the graphical representation by providing a vector input onthe graphical representation 140. In some embodiments, the user canprovide a vector by moving the cursor (or the finger on a touch screendevice) to a different location. As shown, the user has started to movethe cursor 155 to the right of the graph 140, as indicated by the arrow160. Based on the selected location on the graph 140 and the vectorinput, the application of some embodiments generates a color spacetransform that maps each set of color values within a color space toanother set of color values. The generation of the color space transformwill be discussed in more detail below by reference to FIGS. 3, 7, 13,and 19.

At stage 115, the application edits the image 125 according to the colorspace transform that is generated based on the user inputs. As a resultof the cursor moving to the right, the application increases theluminance values of the pixels having relatively high luminance valuesin the image 125. As shown, the area of the image 125 near the sun(i.e., the area of the image 125 with relatively high luminance values)have brightened up, as indicated by the removal of the diagonal linesaround the sun.

The fourth stage 120 illustrates the GUI 100 after the user hascompleted the media editing operation. As shown, the user has moved thecursor 155 further to the right (i.e., almost to the right end of thegraph 140) and has stopped at that new location. This cursor movementprovides a new vector on the graphical representation. Similar to thethird stage 115, the application generates a new color space transformbased on the user inputs and applies the transform to the image 125. Asa result of the cursor movement, the portion of the image 125 that showsthe sky, the sun, and the cloud has brightened up, and is no longerunderexposed.

In the above example, the application continuously generates a new colorspace transform as the cursor moves from one location to anotherlocation. In some embodiments however, the application may generate anew color space transform only when the application detects that thecursor has stopped moving for a predetermined period of time.

FIG. 2 conceptually illustrates a process 200 for editing an image basedon a manipulation on a graphical representation of the color values ofthe image. In some embodiments, the process is performed by amedia-editing application. The process 200 begins by displaying (at 205)a graphical representation of the color values of a particular image onthe GUI of the media-editing application. As mentioned above, theapplication may display different graphical representations that presentcolor data of the image in different graphical forms. For example, acolor channel histogram shows a distribution of color values ofdifferent pixels across the image. A waveform scope shows a distributionof color values of different pixels at each horizontal location of theimage. A vectorscope displays a distribution of chrominance values(e.g., color shades, color intensity, etc.) of different pixels in theimage. The media-editing application of some embodiments may display oneor more forms of graphical representations simultaneously for the image.

Next, the process 200 receives (at 210) a set of inputs on the graphicalrepresentation from the user. In some embodiments, the set of inputsincludes selecting a point on the graph and providing a vector. In somesuch embodiments, the user can provide the vector by dragging a cursoror by dragging the finger across a touch screen device.

After the set of inputs is received, the process 200 determines (at 215)a color space transform based on the received inputs. As mentionedabove, the color space transform maps colors defined within a colorspace in which the color values of the image are defined to a differentset of colors. In some embodiments, the color space transform is analgorithm or a mathematical formula that takes an input color that isdefined within the color space and generates an output color. Beforeapplying the color space transform to an image, the media-editingapplication of some embodiments uses the color space transform toconstruct a lookup table. The lookup table includes a set of inputcolors defined within the color space and a set of corresponding outputcolors that are generated according to the color space transform. Thegeneration and implementation of the color space transform will bediscussed in more detail below by reference to FIG. 3.

The process 200 then applies (at 220) the color space transform to a setof images including the particular the image. If the application hasconstructed a lookup table based on the color space transform, theapplication applies the transform to the image by replacing each colorin the image with the color's corresponding output color in the lookuptable. Otherwise, the media-editing application of some otherembodiments directly applies the color space transform to the image byusing the algorithm or mathematical formula to change the color valuesof the image in real-time.

After applying the color space transform to the image, the process 200updates (at 225) the graphical representation according to theadjustments made to the color values of the image based on the colorspace transform. Next, the process determines (at 230) if more inputsare received on the graphical representation. If more inputs arereceived, the process returns to 215 to determine a transform to thecolor space based on the newly received inputs. The process will cyclethrough operations 215-230 until no more inputs are received on thegraphical representation. Then the process ends.

FIG. 3 illustrates a media-editing application 300 of some embodimentsthat provides a graphical representation of the color values of an imageand allows a user to edit the color values of the image by modifying thegraphical representation. As shown in FIG. 3, the media-editingapplication 300 includes a UI module 305, a color space transformgenerator 310, and a color adjustment engine 315. The color graphs UImodule 305 receives user inputs provided on the graph and passes theinputs to the color space transform generator 310. FIG. 3 illustrates aset of example user inputs 360 on a graphical representation 140. Thegraphical representation 140 is identical to the graphicalrepresentation 140 in the first stage 140 of FIG. 1. In this example,the user selects a location 150 on the graph and provides a vector 320.

The color space transform generator 310 receives the user inputs 360 andgenerates a transform to a color space based on the user inputs. Asmentioned, the color space is a color space in which the color values ofthe image are defined. In some embodiments, the color space isdetermined by the device that captures, displays, or reproduces theimage (e.g., a monitor that displays the image, a printer that printsthe image, etc.). In other embodiments, the color space is specified bya standard or determined by the user who is editing the particularimage.

FIG. 3 also illustrates an example of such a color space transform. Inthis figure, the color space in which the color values of the image aredefined is depicted as a cylinder 325 for illustrative purposes. Thecylinder is defined in a three dimensional space, which can be expressedin terms of Cartesian coordinates (i.e., x, y, and z coordinates),Cylindrical coordinates (i.e., r, φ, and z), or any other coordinatesystem. Accordingly, each color in the cylindrical color space can bespecified by three coordinates in the coordinate system used to definethe color space. In FIG. 3, the area 330 is a representation of thecolors of the image. As the colors of the image only represent a portionof all the colors defined within the color space, the area 330 occupiesonly a fraction of the cylinder 325.

In some embodiments, the color space transform T_(CS) takes each colordefined within the original color space 325 and outputs a differentcolor. For example, the color space transform T_(CS) takes a color 340that is defined in the original color space 325 and outputs a differentcolor 345. As another example, the color space transform T_(CS) may takea color 350 that is defined in the original color space and outputs adifferent color 355. In other words, the transform T_(CS) effectivelywarps the original color space 325 to produce a modified color space 335and maps each color defined within the original color space 325 to adifferent color defined in the modified color space 335.

Different embodiments may use the color space transform differently. Forinstance, some embodiments use the color space transform in real time toconvert an input color value from an image into an output color value.Alternatively, other embodiments use the color space transform generator310 to construct a lookup table that can later be used to map inputcolor values from an image into output color values.

FIG. 3 only illustrates one example color space transform that modifiesthree dimensions of the original color space 325 to produce the modifiedcolor space 335. The color space transform generator 310 may generatedifferent kinds of color space transform based on different usermanipulations of a graphical representation. The various color spacetransforms will be discussed in more detail below by reference to FIGS.7, 13, and 19.

After generating the color space transform T_(CS), the color spacetransform generator 310 sends the color space transform T_(CS) to thecolor adjustment engine 315. The color adjustment engine 315 receives animage and applies the color space transform to the image to produce amodified image.

In some embodiment, the media-editing application also updates thegraphical representation according to the adjustments made to the colorvalues of the image. In this figure, the graphical representation 365 isa representation of the color values of the image after the color spacetransform has been applied to the image. When the graphicalrepresentation is updated, the graphical representation appears to haveshifted and/or skewed in response to the user's input. Specifically, themovements of the graphical representation are consistent with the userinputs on the graph. In this example, the user inputs (i.e., theselected point and the vector) have caused the right portion of thegraphical representation to shift and skew to the right.

In addition to allowing a user to edit an image by directly manipulatingthe graphical representations of the image, some embodiments display aset of color adjustment tools on the image and allow a user to edit thecolor values of the image by controlling the color adjustment tools thatare displayed on the image. FIG. 4 illustrates an example GUI 400 of amedia-editing application of some embodiments that displays a set ofcolor adjustment tools and allows a user to edit an image by controllingthe color adjustment tools on the displayed image. Specifically, theimage-editing operation will be described by reference to the state offour different stages 405, 410, 415, and 420. Each of these stages willbe described in more detail below.

As shown in FIG. 4, the GUI 400 illustrates an image 425 and avectorscope representation 435 of the color values of the image 425. Avectorscope in some embodiments shows the distribution of color of theimage on a circular scale. The color is represented by a series ofconnected points that fall within this scale. The angle around the scalerepresents the hue displayed, with targets indicating the primary colorsof red, green, and blue and the secondary colors of yellow, cyan, andmagenta. The distance from the center of the scale to the outer ringrepresents the saturation of the color being displayed. The center ofthe scale represents zero saturation, while the outer ring representsmaximum saturation.

The image 425 is a picture of a red bicycle (as indicated by thediagonal lines on the bicycle). As mentioned above, the color adjustmenttools do not always appear on the image, but only do so when the coloradjustment tools are activated. The first stage 405 illustrates the GUI400 of the media-editing application before the user has activated thecolor adjustment tools on the displayed image.

The second stage 410 illustrates the GUI 400 after the user hasinitiated an image editing operation by activating the color adjustmenttool on the displayed image 425. Different embodiments allow the user toactivate the color adjustment tools in different manners. In someembodiments, the user may activate the color adjustment tools byselecting a location on the image. In these embodiments, a user canselect a location on the image by placing a cursor on the image andproviding an input (e.g., clicking a button of a cursor controllingdevice, pressing a hot key, etc. or by touching an area of a touchscreen display that displays the image). The media-editing applicationthen determines a position of the user input (the cursor position or theposition of the touch) and corresponds that position to a particularpixel on the image. The position of the user input can be determined byidentifying a set of coordinates (e.g., a horizontal coordinate x and avertical coordinate y) of the user input.

As shown, the user selects the location 450 on the image 425 by placinga cursor 440 on top of an area showing the red bicycle on the image 425.Based on the selected location on the image, the application of someembodiment identifies a particular pixel on the image. In someembodiments, the application uses the color values of the particularpixel to determine a color space transform for the image.

As a result of the selection, the application displays a coloradjustment tool 445 on the image 425. Different embodiments providedifferent color adjustment tools to be displayed on the image. In thisexample, the color adjustment tool 445 simulates a vectorscoperepresentation of the color values of the image. When the coloradjustment tool simulates a color value graphical representation of theimage, the user may provide an additional input (e.g., a vector) on thedisplayed image as if the user is manipulating the actual graphicalrepresentation.

As shown, the color adjustment tool 445 is displayed around the selectedarea on the image 425. Additionally, the application corresponds thecolor values of the particular pixel of the image 425 to a location onthe actual vectorscope representation 435, as indicated by a whitecircle 460 around the location on the vectorscope 435. Although thecolor adjustment tool 445 is shown to be opaque in this example, someembodiments display a translucent UI element on the image in order notto obstruct the view of the image from the user.

The third stage 415 illustrates the GUI 400 after the user has provideda vector input on the displayed image 425. As shown, the user hasprovided a vector 450 by dragging the cursor toward the lower leftcorner of the image 425. As shown in the stage 415, the drag movement ofthe cursor 440 has also created a vector 455 that is applied to theactual vectorscope representation 435. The application then generates acolor space transform that is applied to the image based on user inputs.Specifically, the media-editing application uses the color values of theparticular pixel and the vector input to determine a color spacetransform in the same manner as if the inputs are provided directly onthe actual vectorscope representation. The application then edits theimage and updates the vectorscope representation of the color values ofthe image according to the color space transform. As a result, thepixels that were originally red in color (e.g., the bicycle) have beenchanged to orange (as indicated by the disappearance of the diagonallines of on the bicycle). Also, the vectorscope representation 435 alsoshows the new changes caused by the vector input 455.

The fourth stage 420 illustrates the GUI 400 after the user hascompleted the image editing operation. As shown, the user has moved thecursor 440 toward the bottom of the image 425 and has stopped at thatlocation. This last location of the cursor provides a new vector 465 onthe displayed image 425. Similar to the third stage 415, the applicationgenerates a new color space transform based on the user inputs andapplies the transform to the image 425. As a result, the pixels thatwere originally red (e.g., pixels of the bicycle) have turned into green(as indicated by the crosshatching on the bicycle). This stage alsoillustrates the new changes on the vectorscope representation 435 as aresult of the vector input.

Several more detailed embodiments of the invention are described in thesections below. Sections I to III describe details of editing an imageby directly manipulating different types of graphical representations.Specifically, Section I describes editing an image by directlymanipulating a waveform representation, Section II describes editing animage by directly manipulating a vectorscope representation, and SectionIII describes editing an image by directly manipulating a color channelhistogram representation. Section IV then describes details of editingan image by providing inputs directly on the displayed image. Next,Section V illustrates the software architecture of the media-editingapplication of some embodiments. Finally, Section VI describes anelectronic system that implements some embodiments of the invention.

I. Directly Manipulating a Waveform Representation

As mentioned above, the media-editing application of some embodimentsmay provide one or more different graphical representations in the GUI.Some examples of these different graphical representations are waveformrepresentations, vectorscope representations, and color channelhistogram representations. The operations of editing an image bydirectly manipulating these various graphical representations will bedescribed in more details below. In the examples illustrated below, thevarious graphical representations are shown to be separately displayedin order not to obscure the description with unnecessary details.However, in some embodiments, different graphical representations may bedisplayed simultaneously on the GUI.

FIG. 5 illustrates an example GUI that provides a waveformrepresentation of the color values of an image and allows a user to editthe image by directly manipulating the waveform representation.Specifically, FIG. 5 illustrates the image-editing operation at fourdifferent stages 505, 510, 515, and 520. Each of these stages will bedescribed in more detail below.

The GUI 500 in FIG. 5 illustrates an image 525 and a waveformrepresentation 540 of the color values of the image 525. As mentioned, awaveform scope shows a distribution of luminance values of differentpixels at each horizontal location of the image. Specifically, thewaveform representation is defined along two axes, a vertical axis thatspecifies different luminance values within a color space and ahorizontal axis that specifies different horizontal location of theimage 525. As such, the left most horizontal position on the waveformscope 540 show different luminance values of the left most section ofthe image 525 and the right most horizontal position on the waveformscope 540 shows different luminance values of the right most section ofthe image 525.

The image editing operation will now be described by reference to thestate of the GUI 500 during the four stages 505, 510, 515, and 520. Thefirst stage 505 illustrates the GUI 500 of the media-editing applicationwhen the image 525 is opened. As shown, the image 525 and the waveformrepresentation 540 of the color values of the image 525 are displayed inthe GUI 500. The image 525 is a picture of a red bicycle. As the colorson the right side of the image 525 are generally darker than the colorson the left side of the image 525, the right portion of the waveformrepresentation 540 shows a higher intensity of low luminance values thanthe left portion of the waveform representation 540.

The second stage 510 illustrates the GUI 500 after the user hasinitiated the image editing operation by selecting a location on thewaveform scope 540. The selection of a location on the graph 540 may beperformed by placing a cursor at a desired location on the graph 540 andproviding an input (e.g., clicking on the cursor controlling device,pressing a hot key, etc. or touching an area on a touch screen devicethat displays the graph 540). As shown, the user selects a location 550on the graph 540 by placing a cursor 555 over the location 550 andclicking on the cursor controlling device. Although the user in thisexample has selected a point on the graph 540 that overlaps with an areathat represents the color values of the image 525, the user may alsoselect a point on the graph 540 that does not overlap with such an areain order to initiate the image editing operation.

The third stage 515 illustrates the GUI 500 after the user has provideda vector input on the graphical representation 540. In some embodiments,the user can provide a vector by moving the cursor (or the finger on atouch screen device) to a different location. As shown, the user hasstarted to move the cursor 555 upward, as indicated by the arrow 560.Based on the selected location on the graph 540 and the vector input,the application of some embodiments generates a color space transform.The color transform maps colors defined within the color space in whichthe color values of the image are defined to a different set of colors.

At the stage 515, the application edits the image 525 according to thecolor space transform that is generated based on the user inputs. Asshown, the entire image 525 in this stage appears to be brighter thanthe image 525 in the second stage 510 as a result of the cursormovement.

The fourth stage 520 illustrates the GUI 500 after the user hascompleted the image editing operation. As shown, the user has moved thecursor 555 further upward and has stopped at this new location. Similarto the third stage 515, the application generates a new color spacetransform based on the user inputs and applies the transform to theimage 525. As shown, the entire image 525 in this stage appears in thisstage to be even brighter than the image 525 in the third stage 515 as aresult of the cursor movement.

In the above example, the application continuously generates a new colorspace transform as the cursor moves from one location to anotherlocation. In some embodiments however, the application may generate anew color space transform only when the application detects that thecursor has stopped moving for a predetermined period of time.

FIG. 6 illustrates two different UI implementations (1) and (2) for auser to select a location on the waveform scope. In UI implementation(1), a user may select a location on the waveform scope by placing acursor on the graph and providing an input (e.g., clicking a button of acursor controlling device, pressing a hot key, etc. or by touching anarea of a touch screen display that displays the waveform scope). Insome embodiments, the media-editing application determines a position ofthe user input (the cursor position or the position of the touch) andcorresponds that position to a location on the waveform scope. In someembodiments, the position of the user input can be determined byidentifying a set of coordinates (e.g., a horizontal coordinate x and avertical coordinate y) of the user input. In addition, the user canmanipulate the waveform representation by moving the cursor (or movingthe finger on the touch screen device) to a different position.

In other embodiments, the media-editing application provides severalselectable UI items on the waveform scope. UI Implementation (2)illustrates an example of such an implementation. As shown, theapplication provides several selectable UI items on the waveform scope.The selectable UI items are horizontal slices that represent differenthorizontal sections on the waveform scope. Each horizontal slicecorresponds to a particular luminance value or a particular range ofluminance values. Under this UI implementation, a user can select apoint on the waveform scope by selecting one of the selectable UI items.In addition, the user can manipulate the waveform representation bymoving the selected UI item to a different vertical location on thegraph.

FIG. 7 conceptually illustrates a media-editing application thatprovides a waveform representation of the color values of an image andallows a user to edit the image by directly manipulating the waveformrepresentation. This figure also illustrates several alternativeresulting movements of the waveform representation and an example colorspace transform that is generated from the manipulation of the waveformrepresentation. As shown, the media-editing application 300 includes aUI module 305, the color space transform generator 310, and the coloradjustment engine 315. The UI module 305 receives a set of inputs 725 onthe waveform representation 720 and passes the inputs 725 to the colorspace transform generator 310. The set of inputs 725 includes selectinga location on the waveform representation and providing a vector.

The color space transform generator 310 receives the user inputs 725 andgenerates a transform to a color space based on the user inputs. FIG. 7illustrates an example color space transform T_(CS) resulted from themanipulation of the waveform representation 720 through the tonal graphs730 and 735. The tonal graph is defined along two axes, a horizontalaxis that specifies the original tonal values (input values), with blackon the left and progressively lighter values toward the right, and avertical axis that specifies the changed tonal values (output values),with black on the bottom and lighter values progressing to white at thetop. As such, a tonal graph represents changes to the tonal (luminance)scale of a color space. The tonal graph 730 is a tonal graph of thecolor space prior to applying the transform T_(CS) to the color space.As shown, the tonal curve of the tonal graph 730 is a straight diagonalline that maps each input luminance value to exactly the same luminancevalue in the output. The tonal graph 735 is a tonal graph of the colorspace after applying the color space transform T_(CS). As shown, thecolor space transform warps the tonal curve of the graph 735 such thateach input luminance value is mapped to a different, higher luminanceoutput value.

FIG. 7 only illustrates one example color space transform that isgenerated from the user inputs 725. In this example, the color spacetransform warped the color space to produce the resulting tonal graph735. However, the media-editing application of some embodiments maygenerate different color space transforms that produce different tonalgraphs (not shown) from the same user inputs. In some embodiments, theuser may specify a particular color space transform by providingadditional input, such as a hot key or an addition mouse click.

Different color space transforms also cause the waveform representationto be modified differently. FIG. 7 shows three different examplemodifications to the waveform representation based on three differentcolor space transforms resulted from the user inputs 725. In example(1), the entire waveform representation shifts upward. In other words,the luminance value of every pixel is increased by the same amount. Inexample (2), only the portion of the waveform representation around thepoint selected by the user skews upward. In example (3) the entirewaveform representation shifts and skews upward, where the portion ofthe representation closer to the point selected by the user is moved bya larger amount than the portion further away from the selected point.

To generate a color space transform from the waveform manipulation, theapplication of some embodiments first identifies a particular luminancevalue that corresponds to the selected location on the waveform scopeand uses the vector input to determine an adjustment to the particularluminance value. The application then uses the particular luminancevalue adjustment to determine adjustments to the other luminance valuesdefined within the color space. In some embodiments, the media-editingapplication constructs a blended Gaussian curve to determine amounts ofadjustment to be applied to each luminance value based on the userinputs. The Gaussian curve is a symmetrical bell shaped curve thatidentifies an amount of adjustment to be applied to each color valuerepresented in the graphical representation. FIG. 8 conceptuallyillustrates, in two stages, an example Gaussian curve that is generatedbased on a set of user inputs provided on a waveform representation. Asshown, FIG. 8 includes a waveform representation 820 of the color valuesof an image (not shown) and a graph 830. The graph 830 is defined alongtwo axes, a horizontal axis that specifies luminance values, withsmaller luminance values on the left and progressively higher luminancevalues toward the right, and a vertical axis that specifies differentamounts of adjustments. In the first stage 805, since the user has notprovided any inputs on the waveform representation, the graph 830 showsa flat line, indicating that no change is applied to any luminancevalues.

When the user selects a location 825 on the waveform representation, themedia-editing application translates the selected location 825 to aparticular luminance value and identifies a horizontal location 835 onthe graph 830 that corresponds to the particular luminance value.

In the second stage 810, the user has moved the cursor upward to providea vector 845 on the waveform representation 820. Based on the selectedpoint and the cursor movement, the media-editing application of someembodiments generates a Gaussian curve 850 on the graph 830. A Gaussiancurve is a symmetrical bell-shaped curve that is generated using aGaussian function

${{f(x)} = {a\; ^{- \frac{{({x - b})}^{2}}{2c^{2}}}}},$

whereas parameter “a” represents the height of the curve's peak (i.e.,the highest point on the Gaussian curve), parameter “b” represents theposition of the center of the peak, and parameter “c” controls the widthof the “bell”. The application determines the center of the peak (i.e.,parameter “b”) based on the user's selected location on the histogram825. The application also determines the height of the peak (i.e.,parameter “a”) based on the user's vector input 845 on the histogram825. Different embodiments construct the Gaussian curves with differentwidths by using different values for the parameter “c” in the Gaussianfunction. For example, the application of some embodiments may use alarger “c” value (e.g., 0.8) to construct a Gaussian curve with a widewidth and the application of other embodiments may use a smaller “c”value (e.g., 0.2) to construct a Gaussian curve with a narrow width.

The application may also blend this newly constructed Gaussian curvewith any existing Gaussian curves that were constructed based onpreviously provided input to create a blended Gaussian curve. FIG. 9illustrates an example blended Gaussian curve 920 on the graph 905. Asshown in the figure, two different Gaussian curves 910 and 915 have beencreated on the graph 905. The application of some embodiments blend thetwo curves 910 and 915 by adding the values of the two curves to createthe blended curve 920. Using this blended Gaussian curve, theapplication identifies different amounts of adjustment to be applied todifferent color values in the image.

In some embodiments, the application translates the amount of adjustmentfor each luminance value to a set of adjustments that is applied to thedifferent color component values defined within the color space. Forexample, when the color values of the image are defined in the RGB colorspace (wherein each color is represented by a combination of a redvalue, a green value, and a blue value), the application may translatean adjustment to each luminance value to a set of adjustments to a redvalue, a green value, and a blue value so that the adjustments can bedirectly applied to the color values of the image.

FIG. 10 conceptually illustrates a process 1000 that is performed by amedia-editing application of some embodiments for editing an image basedon a direct manipulation on a waveform representation of the colorvalues of the image. The process begins by displaying (at 1005) awaveform representation of the color values of a particular image on theGUI of the media-editing application.

Next, the process receives (at 1010) a selection of a location on thewaveform representation. As mentioned above, a user may select alocation on the waveform representation by placing a cursor at a desiredlocation on the waveform representation and providing an input (e.g.,clicking on the cursor controlling device, pressing a hot key, etc. ortouching an area on a touch screen device that displays the waveformrepresentation).

After the process has received a selected location on the waveformrepresentation, the process translates (at 1015) the selected locationto a luminance value. In some embodiments, the luminance value isdetermined based on the vertical position (e.g., the vertical coordinatey) of the selected location on the waveform representation.

The process then receives (at 1020) a vector input from the user on thewaveform representation. In some embodiments, the user may provide thevector input by dragging a cursor or moving a finger on a touch screendevice. Next, the process translates (at 1025) the vector input to achange of the luminance value. In some embodiments, the direction of thevector is used to determine an increase or a decrease of the luminancevalue and the magnitude of the vector is used to determine an amount ofchange to the luminance value.

Then, the process determines (at 1030) a transform to a color spacebased on the change of the luminance value. In some embodiments, thecolor space transform is determined based on a Gaussian curve that isconstructed in a manner as illustrated above by reference to FIG. 8. Insome embodiments, the Gaussian curve is constructed by using the changeof the color value as a parameter.

After the color space transform is determined, the process applies (at1035) the color space transform to a set of images including theparticular image. In some embodiments, the set of images are pictureframes of a video clip. In other embodiments, the set of images onlyincludes the particular image.

Next, the process updates (at 1040) the waveform representationaccording to the adjustments to the particular image. As mentionedabove, different color space transforms cause different modifications tothe waveform representation. Depending on the particular color spacetransform that was generated, the waveform representation may be updatedin different manners. The process then determines (at 1045) if anothervector input is received on the waveform representation. If anothervector input is received, the process returns to 1025 to translate thenew vector to a change of the color value, and will cycle throughoperations 1025-1045 until no more vector inputs are received on thewaveform representation. If no more vector inputs are received, theprocess determines (at 1050) if another location on the waveformrepresentation is selected. If another location on the waveformrepresentation is selected, the process returns to 1015 to translate theselected location to a color value, and will cycle through operations1015-1050 until no more location is selected on the waveformrepresentation. Then the process ends.

II. Directly Manipulating a Vectorscope Representation

Section I above described editing an image by directly manipulating awaveform representation of the image. This section describes details ofediting an image by directly manipulating a vectorscope representationof the image.

FIG. 11 illustrates an example GUI that provides a vectorscoperepresentation of the color values of an image and allows a user to editthe image by directly manipulating the vectorscope representation.Specifically, FIG. 11 illustrates the image-editing operation at fourdifferent stages 1105, 1110, 1115, and 1120. Each of these stages willbe described in more detail below.

As shown in FIG. 11, the GUI 1100 illustrates an image 1125 and avectorscope representation 1140 of the color values of the image 1125.As mentioned, a vectorscope shows a distribution of chrominance values(e.g., color shades, color intensity, etc.) of different pixels in theimage. Specifically, a vectorscope shows the distribution of thechrominance values using a two-dimensional polar graph. As such, eachpoint in the vectorscope can be defined by a set of Polar coordinates(r, θ). The degree “θ” specifies color shades. Starting at the twelveo'clock position of the vectorscope going clockwise, the different colorshades that are represented in the vectorscope include magenta, blue,cyan, green, yellow, and red. The radius “r” of the vectorscopespecifies the intensity (i.e., the saturation) of the color. In otherwords, a position that is farther away from the center of thevectorscope corresponds to a more intensed and saturated color than aposition that is closer to the center.

The image editing operation will now be described by reference to thestate of the GUI 1100 during the four stages 1105, 1110, 1115, and 1120.The first stage 1105 illustrates the GUI 1100 of the media-editingapplication when the image 1125 is opened. As shown, the image 1125 andthe vectorscope representation 1140 of the color values of the image1125 are displayed in the GUI 1100. The image 1125 is a picture of abicycle that is red in color, as indicated by the diagonal lines on topof the bicycle. As the dominant colors of the image 1125 are black,white, red, blue and yellow, the vectorscope representation 1140 shows ahigh concentration in the middle, the red, the yellow, and the bluesections.

The second stage 1110 illustrates the GUI 1100 after the user hasinitiated the image editing operation by selecting a location on thevectorscope representation 1140. The selection of a location on thevectorscope representation 1140 may be performed by placing a cursor ata desired location on the vectorscope representation 1140 and providingan input (e.g., clicking on the cursor controlling device, pressing ahot key, etc. or touching an area on a touch screen device that displaysthe vectorscope representation 1140). As shown, the user selects alocation 1150 on the vectorscope representation 1140 by placing a cursorK55 over the location 1150. The selected point 1150 corresponds to ahighly saturated red color. Although the user in this example hasselected a location on the graph 1140 that overlaps with an area thatrepresents the color values of the image 1125, the user may also selecta location on the graph 1140 that does not overlap with such an area inorder to initiate the image editing operation.

The third stage 1115 illustrates the GUI 1100 after the user hasprovided a vector input on the vectorscope representation 1140. In someembodiments, the user can provide a vector input by moving the cursor(or the finger on a touch screen device) to a different location. Asshown, the user has started to move the cursor 1155 toward the bottomleft of the graph (toward the orange section of the vectorscope), asindicated by the arrow 1160. Based on the selected point on the graphand the vector input, the application of some embodiments generates acolor space transform. The color space transform maps colors definedwithin the color space in which the color values of the image aredefined to a different set of colors.

As shown in the stage 1115, the application edits the image 1125according to the color space transform that is generated based on theuser inputs. In particular, the pixels in the image 1125 with variousshades of red color have turned into orange (as indicated by thedisappearance of the diagonal lines), as a result of the user'smanipulation of the vectorscope representation 1140.

The fourth stage 1120 illustrates the GUI 1100 after the user hascompleted the image editing operation. As shown, the user has moved thecursor 1155 further down toward the green section of the vectorscope. Insome embodiments, the application generates a new color space transformbased on the new movement of the cursor 1155. This new color spacetransform is then applied to the image 1125. As shown, the pixels in theimage 1125 that were originally red and yellow have turned green (asindicated by the cross hatching on the bicycle), as a result of thecursor movement on the vectorscope representation 1140.

In the above example, the application continuously generates a new colorspace transform as the cursor moves from one location to anotherlocation. In some embodiments however, the application may generate anew color space transform only when the application detects that thecursor has stopped moving for a predetermined period of time.

FIG. 12 illustrates two different UI implementations (1) and (2) for auser to select a point on the vectorscope. In UI implementation (1), auser may select a location on the vectorscope by placing a cursor on thegraph and providing an input (e.g., clicking a button of a cursorcontrolling device, pressing a hot key, etc., or by touching an area ofa touch screen display that displays the vectorscope). In theseembodiments, the media-editing application determines the position ofthe user input (the cursor position or the position of the touch) andcorresponds that position to a location on the vectorscope. In someembodiments, the position of the user input can be determined byidentifying a set of coordinates (e.g., a horizontal coordinate x and avertical coordinate y) of the user input. In addition, the user canmanipulate the vectorscope representation by moving the cursor (ormoving the finger on the touch screen device) to a different position.

In other embodiments, the media-editing application provides severalselectable UI items on the vectorscope. UI Implementation (2)illustrates an example of such an implementation. As shown, theapplication provides several selectable UI items on the vectorscope. Theselectable UI items represent different sections on the vectorscope.Each UI item corresponds to a particular set of chrominance valueswithin a color space. Under this UI implementation, a user can select apoint on the vectorscope by selecting one of the selectable UI items. Inaddition, the user can manipulate the waveform representation by movingthe selected UI item to a different location on the vectorscope.

FIG. 13 conceptually illustrates a media-editing application thatprovides a vectorscope representation of the color values of an imageand allows a user to edit the image by directly manipulating thevectorscope representation. This figure also illustrates severalalternative resulting movements of the vectorscope representation and anexample color space transform that is generated from the manipulation ofthe vectorscope representation. As shown, the media-editing application300 is identical to the application 300 of FIG. 3. The application 300includes a UI module 305, the color space transform generator 310, andthe color adjustment engine 315. The UI module 305 receives a set ofinputs 1325 on the vectorscope representation 1320 and passes the inputs1325 to the color space transform generator 310. The set of inputs 1325includes selecting a point on the vectorscope representation andproviding a vector.

The color space transform generator 310 receives the user inputs 1325and generates a transform to a color space based on the user inputs. Inthis example, since the vectorscope representation 1320 presents twodimensions (e.g., color hue and color intensity) of the color values ofthe image, the color space transform is also two-dimensional. As shownin the figure, the original color space is depicted as a cylinder 1330for illustrative purposes. Each dimension of the cylinder 1330 (e.g.,the radius, the circumference, and the height) corresponds to adimension of the color space (e.g., the Y channel, Cr channel, and Cbchannel in the YCrCb color space). A cylinder 1335 depicts the colorspace after the color space has been warped or modified by the colorspace transform T_(CS). In this example, the color space transformT_(CS) modifies two dimensions of the color space.

FIG. 13 only illustrates one example color space transform that isgenerated from the user inputs 1325. In this example, the color spacetransform warped the color space to produce the resulting color space1335. However, the media-editing application of some embodiments maygenerate different color space transforms from the same user inputs. Insome embodiments, the user may specify a particular color spacetransform by providing additional input, such as a hot key or anaddition mouse click.

Different color space transforms also cause the vectorscoperepresentation to be modified differently. FIG. 13 shows three differentexample modifications to the vectorscope representation based on threedifferent color space transforms resulted from the user inputs 1325. Inexample (1), the entire vectorscope representation shifts to the left.In other words, a same amount of changes is applied to each color. Inexample (2), only the portion of the vectorscope representation aroundthe point selected by the user skews to the left. In example (3), theentire vectorscope representation shifts and skews to the left, wherethe portion of the representation closer to the point selected by theuser is moved by a larger amount than the portion further away from theselected point.

To generate a color space transform from the vectorscope manipulation,the application of some embodiments first identifies a particular colorvalue that corresponds to the selected location on the vectorscope anduses the vector input to determine an adjustment to the particular colorvalue. The application then uses the particular color value adjustmentto determine adjustments to the other color values defined within thecolor space. In some embodiments, the media-editing applicationdetermines the adjustments to the other color values based on how closethe other color values are to the particular color value on thevectorscope. That is, larger adjustments will be applied to the colorvalues that are closer to the particular color value on the vectorscopeand smaller adjustments will be applied to the color values that arefarther away from the particular color value on the vectorscope. FIG. 14illustrates an example of this implementation. As shown in example (1),after a user has selected a location 1410 and provided a vector 1415 ona vectorscope 1405, the application creates a falloff region 1420 on thevectorscope 1405. Specifically, the falloff region 1420 has a center atthe selected location 1410. The application determines an amount ofadjustment to be applied to each color value based on the location ofthe color values within the falloff region. In these embodiments, theapplication applies 100% of the particular color value adjustment to thecolor value that is located at the center of the falloff region 1420.The application then applies only a percentage of the particular colorvalue adjustment to the other color values, based on how close the colorvalue is to the center of the falloff region. For example, theapplication may apply 90% of the particular color value adjustment to acolor that is located within 10% of the total radius in the falloffregion. The application may also apply no adjustment to color valuesthat are located outside of the falloff region.

In addition to using the falloff region, the media-editing applicationof some embodiments uses a fluid model to determine the adjustments tothe other color values. Under this fluid model, the application dividesthe vector input that is provided on a vectorscope into several smallervectors. The application then determines the adjustments to the colorvalues as if the smaller vectors are provided to the vectorscope one ata time. Example (2) of FIG. 14 illustrates such an implementation afterthe user has provided a vector 1415 on the vectorscope 1405. As shown,the application divides the vector input 1415 into several smallervectors 1425-1440 and creates a falloff region (1445-1460) for each ofthe smaller vectors. The application then determines the adjustment tothe color values in the same manner as described above as if the smallervectors 1425-1440 are provided on the vectorscope consecutively. Bysplitting the vector into several smaller vectors, the applicationprovides a smoother adjustment to the color values.

FIG. 15 conceptually illustrates a process 1500 for splitting a vectorinput on a vectorscope under the fluid model. In some embodiments, theprocess 1500 is performed by the media-editing application immediatelyafter a user provides a vector input on a vectorscope representation ofan image's color values. Thus, the process begins by receiving (at 1505)a vector input V with a start point P1 and an end point P2 on thevectorscope representation. Then the process determines (at 1510) thenumber of small vectors for the vector input.

Different embodiments use different technique to determine the number ofsmall vectors for a vector input. For instance, the application of someembodiments splits the vector based on the magnitude (i.e., the length)of the vector. In these embodiments, the application determines a numberof small vectors by determining a smallest integer multiplier of apredetermined number (e.g., 0.1) that would yield a number equal to orgreater than the magnitude of the vector. For example, if the magnitudeof the vector is 0.5, the smallest integer multiplier of 0.1 (thepredetermined number) that would yield a number equal to or greater than0.5 would be 5.

Next, the process creates (at 1515) a first small vector with a startpoint at P1 and an end point at P1+V/(number of small vectors). Theprocess then determines (at 1520) if the number of small vectors equalsto 1. If the number of small vectors equals to 1, the process ends.However, if the number of small vectors is greater than 1 (i.e., or notequal to 1), the process continues to 1525 and creates another smallvector with a start point at the end point of the last small vector andan end point at the new start point+V/(number of small vectors).

Then, the process determines (at 1530) if more small vectors need to becreated. If no more small vectors are needed, the process ends. However,if more small vectors are needed, the process returns to 1525 to createanother small vector, and will cycle through operations 1525-1530 untilno more small vectors are needed.

FIG. 16 conceptually illustrates a process 1600 that is performed by amedia-editing application of some embodiments for editing an image basedon a direct manipulation on a vectorscope representation of the colorvalues of the image. The process begins by displaying (at 1605) avectorscope representation of the color values of a particular image onthe GUI of the media-editing application.

Next, the process receives (at 1610) a selection of a location on thevectorscope representation. As mentioned above, a user may select alocation on the vectorscope representation by placing a cursor at adesired location on the waveform representation and providing an input(e.g., clicking on the cursor controlling device, pressing a hot key,etc. or touching an area on a touch screen device that displays thevectorscope representation).

After the process has received a selected location on the vectorscoperepresentation, the process translates (at 1615) the selected locationto a set of chrominance values. In some embodiments, the set ofchrominance values is determined based on the horizontal and verticalpositions (e.g., the horizontal and vertical coordinates) of theselected location on the vectorscope representation.

The process then receives (at 1620) a vector input from the user on thevectorscope representation. In some embodiments, the user may providethe vector input by dragging a cursor or moving a finger on a touchscreen device. Next, the process translates (at 1625) the vector inputto a change of the set of chrominance values. In some embodiments, thedirection of the vector is used to determine an increase and/or adecrease of different color channel values and the magnitude of thevector is used to determine the amount of change to the set ofchrominance values.

Then, the process determines (at 1630) a transform to a color spacebased on the change of the set of chrominance values. After the colorspace transform is determined, the process applies (at 1635) the colorspace transform to a set of images including the particular image. Insome embodiments, the set of images are picture frames of a video clip.In other embodiments, the set of images only include the particularimage.

Next, the process updates (at 1640) the vectorscope representationaccording to the adjustments to the particular image. As mentionedabove, different color space transforms cause different modifications tothe vectorscope representation. Depending on the color space transformthat was generated, the vectorscope representation may be updated indifferent manners.

The process then determines (at 1645) if another vector is received onthe vectorscope representation. If another vector input is received, theprocess returns to 1625 to translate the newly received vector input toa change of the set of chrominance values, and will cycle throughoperations 1625-1645 until no more vector inputs are received on thevectorscope representation. If no more vector inputs are received, theprocess determines (at 1650) if another location on the vectorscoperepresentation is selected. If another location on the vectorscoperepresentation is selected, the process returns to 1615 to receiveanother selection, and will cycle through operations 1615-1650 untilthere is no more selection provided on the vectorscope representation.Then the process ends.

When the application applies the fluid model to determine the colorspace transform, the application may divide the vector input intoseveral small vectors as described above, and perform operations1625-1645 repeatedly until all small vectors are processed.

III. Directly Manipulating a Histogram Representation

Sections I and II above described editing an image by directlymanipulating a waveform and a vectorscope representation of the image.This section describes details of editing an image by directlymanipulating a color channel histogram representation of the image.

A. Manipulating a Histogram Representation

FIG. 17 illustrates an example GUI that provides a color channelhistogram representation of the color values of an image and allows auser to edit the image by directly manipulating the color channelhistogram representation. Specifically, FIG. 17 illustrates theimage-editing operation at four different stages 1705, 1710, 1715, and1720. Each of these stages will be described in more detail below.

As shown in FIG. 17, the GUI 1700 includes an image 1725 and a colorchannel histogram representation 1740 of the color values of the image1725. In this example, the color channel histogram 1740 shows adistribution of luminance values of different pixels across the image.Specifically, the histogram representation is defined along two axes, ahorizontal axis that specifies luminance values and a vertical axis thatspecifies different numbers of pixels. Thus, each horizontal location ofthe histogram shows the number of pixels within the image having thecorresponding luminance value. Although the histogram representation inthis example shows a distribution of luminance values of differentpixels across an image, the media-editing application of someembodiments may provide a histogram representation that shows adistribution of other channel values (e.g., red channel values, bluechannel values, or green channel values, when the colors of the imageare defined within a RGB color space, or Y channel) of different pixelsof an image.

The image editing operation will now be described by reference to thestate of the GUI 1700 during the four stages 1705, 1710, 1715, and 1720.The first stage 1705 illustrates the GUI 1700 of the media-editingapplication when the image 1725 is opened. As shown, the image 1725 andthe color channel histogram representation 1740 of the color values ofthe image 1725 are displayed in the GUI 1700. The image 1725 is apicture of a red bicycle. As the image 1725 has more dark areas thanbright areas, the color channel histogram 1740 shows a higherconcentration of pixels with color values that have low luminancevalues.

The second stage 1710 illustrates the GUI 1700 after the user hasinitiated the image editing operation by selecting a location on thecolor channel histogram representation 1740. The selection of a locationon the histogram 1740 may be performed by placing a cursor at a desiredlocation on the histogram 1740 and providing an input (e.g., clicking onthe cursor controlling device, pressing a hot key, etc. or touching anarea on a touch screen device that displays the histogram 1740). Asshown, the user selects a location 1750 on the histogram 1740 by placinga cursor 1755 over the location 1750. Although the user in this examplehas selected a location on the graph 1740 that overlaps with an areathat represents the color values of the image 1725, the user may alsoselect a location on the graph 1740 that does not overlap with such anarea in order to initiate the image editing operation.

The third stage 1715 illustrates the GUI 1700 after the user hasprovided a vector on the histogram representation 1740. In someembodiments, the user can provide a vector by moving the cursor (or thefinger on a touch screen device) to a different location. As shown, theuser has started to move the cursor 1755 to the right, as indicated bythe arrow 1760. Based on the selected location 1750 on the histogram1740 and the movement of the cursor 1755, the application of someembodiments generates a color space transform that will be applied tothe image 1725.

As shown in the stage 1715, the application edits the image 1725according to the color space transform that is generated based on theuser inputs. In particular, the entire image 1725 in this stage appearsto be brighter than the image 1725 in the second stage 1710 as a resultof the cursor movement.

The fourth stage 1720 illustrates the GUI 1700 after the user hascompleted the image editing operation. As shown, the user has moved thecursor 1755 further to the right. In some embodiments, the applicationgenerates a new color space transform based on the new movement of thecursor 1755. This new color space transform is then applied to the image1725. As shown, the entire image 1725 appears to be even brighter thanthe image 1725 in the third stage 1715 as a result of the cursormovement.

In the above example, the application continuously generates a new colorspace transform as the cursor moves from one location to anotherlocation. In some embodiments however, the application may generate anew color space transform only when the application detects that thecursor has stopped moving for a predetermined period of time.

FIG. 18 illustrates two different UI implementations (1) and (2) for auser to select a location on the color channel histogram. In UIimplementation (1), a user may select a location on the color channelhistogram by placing a cursor on the graph and providing an input (e.g.,clicking a button of a cursor controlling device, pressing a hot key,etc. or by touching an area of a touch screen display that displays thehistogram). In these embodiments, the media-editing applicationdetermines a position of the user input (the cursor position or theposition of the touch) and corresponds that position to a location onthe histogram. In some embodiments, the position of the user input canbe determined by identifying a set of coordinates (e.g., a horizontalcoordinate x and a vertical coordinate y) of the user input. Inaddition, the user can manipulate the histogram representation by movingthe cursor (or moving the finger on the touch screen device) to adifferent position.

In other embodiments, the media-editing application provides severalselectable UI items on the histogram. UI Implementation (2) illustratesan example of such an implementation. As shown, the application providesseveral selectable UI items on the histogram representation. Theselectable UI items are vertical slices that represent differentvertical sections on the histogram. Each vertical slice corresponds to aparticular luminance value or a particular range of luminance values.Under this UI implementation, a user can select a location on thehistogram representation by selecting one of the selectable UI items. Inaddition, the user can manipulate the histogram representation by movingthe selected UI item to a different horizontal location on the graph.

FIG. 19 conceptually illustrates a media-editing application thatprovides a color channel histogram representation of the color values ofan image and allows a user to edit the image by directly manipulatingthe histogram representation. This figure also illustrates severalalternative resulting movements of the histogram representation and anexample color space transform that is generated from the manipulation ofthe histogram representation. As shown, the media-editing application300 is identical to the application 300 of FIG. 3. The application 300includes a UI module 305, the color space transform generator 310, andthe color adjustment engine 315. The UI module 305 receives a set ofinputs 1925 on the color channel histogram representation 1920 andpasses the inputs 1925 to the color space transform generator 310. Theset of inputs 1925 includes selecting a location on the histogramrepresentation and providing a vector.

The color space transform generator 310 receives the user inputs 1925and generates a transform to a color space based on the user inputs. Inthis example, since the histogram representation 1920 only presents onedimension (e.g., the luminance channel) of the color values of theimage, the color space transform is also one-dimensional. FIG. 19illustrates an example color space transform T_(CS) resulted from themanipulation of the histogram representation 1920 through the tonalgraphs 1930 and 1935. The tonal graph is defined along two axes, ahorizontal axis that specifies the original tonal values (input values),with black on the left and progressively lighter values toward theright, and a vertical axis that specifies the changed tonal values(output values), with black on the bottom and lighter values progressingto white at the top. As such, a tonal graph represents changes to thetonal (luminance) scale of a color space. The tonal graph 1930 is atonal graph of the color space prior to applying the color spacetransform T_(CS). As shown, the tonal curve of the tonal graph 1930 is astraight diagonal line that maps each input luminance value to exactlythe same luminance value in the output. The tonal graph 1935 is a tonalgraph of the color space after applying the color space transformT_(CS). As shown, the color space transform warps the tonal curve of thegraph 1935 such that each input luminance value is mapped to adifferent, higher luminance output value.

FIG. 19 only illustrates one example color space transform that isgenerated from the user inputs 1925. In this example, the color spacetransform warped the color space to produce the resulting tonal graph1935. However, the media-editing application of some embodiments maygenerate different color space transforms that produce different tonalgraphs (not shown) from the same user inputs. In some embodiments, theuser may specify a particular color space transform by providingadditional input, such as a hot key or an addition mouse click.

Different color space transforms also cause the histogram representationto be modified differently. FIG. 19 shows three different examplemodifications to the histogram representation based on three differentcolor space transforms resulted from the user inputs 1925. In example(1), the entire histogram representation shifts to the right. In otherwords, the luminance value of every pixel is increased by the sameamount. In example (2), only the portion of the histogram representationaround the location selected by the user skews to the right. In example(3) the entire histogram representation shifts and skews to the right,where the portion of the representation closer to the location selectedby the user is moved by a larger amount than the portion further awayfrom the selected location.

To generate a color space transform from the histogram manipulation, theapplication of some embodiments first identifies a particular colorvalue of a color channel that corresponds to the selected location onthe histogram representation and uses the vector input to determine anadjustment to the particular color value. The application then uses theparticular color value adjustment to determine adjustments to the othercolor values of the color channel defined within the color space. Insome embodiments, the media-editing application constructs a blendedGaussian curve to determine amounts of adjustment to be applied to eachcolor value based on the user inputs. The Gaussian curve is asymmetrical bell shaped curve that identifies the amount of adjustmentto be applied to each color value represented in the graphicalrepresentation. FIG. 20 conceptually illustrates an example Gaussiancurve that is generated based on a set of user inputs on a color channelhistogram representation. As shown, FIG. 20 includes a luminancehistogram representation 2020 of the color values of an image (notshown) and a graph 2030. The graph 2030 is defined by two axes, ahorizontal axis that specifies luminance values, with smaller luminancevalues on the left and progressively higher luminance values toward theright, and a vertical axis that specifies different amounts ofadjustments that are applied to the different luminance values. In thefirst stage 2005, since the user has not provided any inputs on thehistogram representation, the graph 2030 shows a flat line, indicatingthat no change is applied to any luminance values.

When the user selects a location 2025 on the histogram representation,the media-editing application translates that location to a particularluminance value and identifies a horizontal location 2035 on the graph2030 that corresponds to the particular luminance value.

In the second stage 2010, the user has provided a vector 2045 on thehistogram 2020 (e.g., by moving the cursor). Based on the selectedlocation and the cursor movement, the media-editing application of someembodiments generates a Gaussian curve 2050 on the graph 2030. AGaussian curve is a symmetrical bell-shaped curve that is generatedusing a Gaussian function

${{f(x)} = {a\; ^{- \frac{{({x - b})}^{2}}{2c^{2}}}}},$

whereas parameter “a” represents the height of the curve's peak (i.e.,the highest point on the Gaussian curve), parameter “b” represents theposition of the center of the peak, and parameter “c” controls the widthof the “bell”. The application determines the center of the peak (i.e.,parameter “b”) based on the user's selected location on the histogram2025. The application also determines the height of the peak (i.e.,parameter “a”) based on the user's vector input 2045 on the histogram2025. Different embodiments construct the Gaussian curves with differentwidths by using different values for the parameter “c” in the Gaussianfunction. For example, the application of some embodiments may use alarger “c” value (e.g., 0.8) to construct a Gaussian curve with a widewidth and the application of other embodiments may use a smaller “c”value (e.g., 0.2) to construct a Gaussian curve with a narrow width.

The application may also blend this newly constructed Gaussian curvewith any existing Gaussian curves that were constructed based onpreviously provided input to create a blended Gaussian curve in the samemanner as described above by reference to FIG. 9. Using this blendedGaussian curve, the application identifies different amounts ofadjustment to be applied to different color values in the image.

The application of some embodiments may employ a function to blend twoor more basic bumps. For instance, the application may use a function

${{B(t)} = {\sum\limits_{i = 1}^{n}{{f\left( {1 - \frac{t - {bc}_{i}}{{bw}_{i}}} \right)}{bh}_{i}}}},$

whereas B(t) is the composite bump from blending n bumps, t is ahorizontal location of the tonal adjustment graph, be is the baselinelocation of each basic bump, bw is the width of each basic bump, and bhis the height of each basic bump. That is, the composite bump'sadjustment value for a particular horizontal location of the tonaladjustment graph is a sum of adjustment values of the basic bumps thatare weighted for the particular horizontal location by a weightingfunction.

Different embodiments use different weighting functions. For instance,the application of some embodiments may use a Gaussian function, asmoothing function, a liner function, etc. More specifically, theapplication of some embodiments may use a function,

f(x)=6x ⁵−15x ⁴+10x ³

or a function,

f(x)=x ²*(3−2x),

whereas in each of these two functions, x is

$1 - {\frac{t - {bc}_{i}}{{bw}_{i}}.}$

Moreover, the application of some embodiments employs a simple cubicHermite interpolation technique or Perlin's smooth step to computeadjustment values when the adjustment values and horizontal location areof floating type. That is, when the resulting adjustment values of thecomposite bump do not have sufficient granularity, the application ofthese embodiments interpolates to generate intermediate adjustmentvalues that have sufficient granularity.

In some embodiments, the application translates the adjustment for eachluminance value to a set of adjustments that is applied to the differentcolor channels defined within the color space. For example, when thecolor values of the image are defined within the RGB color space, theapplication may translates an adjustment for each luminance value todifferent adjustments to the red channel, the green channel, and theblue channel so that the adjustments can be directly applied to thecolor values of the image.

The example in FIG. 20 illustrates a Gaussian curve that is generated bya set of inputs provided on a luminance histogram representation.However, the application may use similar techniques to construct aGaussian curve when the user provides a set of inputs on a differentcolor channel histogram representation. For example, when a userprovides a set of inputs on a red channel histogram representation, theapplication constructs a Gaussian curve that maps each red channel valueto a different adjustment.

FIG. 21 conceptually illustrates a process 2100 that is performed by amedia-editing application of some embodiments for editing an image basedon a direct manipulation on a color channel histogram representation ofthe color values of the image. The process begins by displaying (at2105) a color channel histogram representation of the color values of aparticular image on the GUI of the media-editing application.

Next, the process receives (at 2110) a selection of a location on thehistogram representation. As mentioned above, a user may select alocation on the histogram representation by placing a cursor at adesired location on the histogram representation and providing an input(e.g., clicking on the cursor controlling device, pressing a hot key,etc. or touching an area on a touch screen device that displays thehistogram representation).

After the process has received a selected location on the histogramrepresentation, the process translates (at 2115) the selected locationto a color value (e.g., a luminance value, a red channel value, a greenchannel value, etc.). In some embodiments, the color value is determinedbased on the horizontal position of the selected location on thehistogram representation.

The process then receives (at 2120) a vector input from the user on thehistogram representation. In some embodiments, the user may provide thevector input by dragging a cursor or moving a finger on a touch screendevice. Next, the process translates (at 2125) the vector input to achange of the color value. In some embodiments, the direction of thevector is used to determine an increase or a decrease of the color valueand the magnitude of the vector is used to determine an amount of changeto the color value.

Then, the process determines (at 2130) a transform to the color spacebased on the change of the color value. In some embodiments, the colorspace transform is determined by a Gaussian curve as illustrated aboveby reference to FIG. 20. In some embodiments, the Gaussian curve isconstructed by using the change of the color value as a parameter.

After the color space transform is determined, the process applies (at2135) the color space transform to a set of images including theparticular image. In some embodiments, the set of images are pictureframes of a video clip. In other embodiments, the set of images onlyincludes the particular image.

Next, the process updates (at 2140) the histogram representationaccording to the adjustments to the particular image. As mentionedabove, different embodiments generate different color space transformsbased on the same set of user inputs. Depending on the color spacetransforms that was generated, the histogram representation may beupdated in different manners. The process then determines (at 2145) ifanother vector input is received on the histogram representation. Ifanother vector input is received, the process returns to 2125 totranslate the vector input to a change of the color value, and willcycle through operations 2125-2145 until there is no more vector inputsprovided on the histogram representation. If there are no more vectorinputs, the process determines (at 2150) if another location on thehistogram representation is selected. If there is another location onthe histogram representation is selected, the process returns to 2110 totranslate the newly selected location to a color value, and will cyclethrough operations 2115-2150 until there is no more selection providedon the waveform representation. Then the process ends.

In some embodiments, the generation of a color space transform is onlydependent on a set of user inputs provided on the graphicalrepresentation of the color values of an image, and is not dependent onthe color values of the image. Thus, when two identical sets of userinputs are provided on the graphical representations of the color valuesof two different images, the application would generate two identicalcolor space transforms. FIG. 22 illustrates such an example. As shown inFIG. 22, two identical user inputs 2215 and 2220 have been provided ontwo color channel histogram representations 2205 and 2210. The histogramrepresentations 2205 and 2210 represent color values of two differentimages (not shown). However, when the same user inputs are provided onthe histogram representations 2205 and 2210, the application creates twoidentical color space transforms.

Some embodiments of the application allow a user to provide more thanone set of user inputs to manipulate the graphical representation of thecolor values of an image. FIG. 23 illustrates an example GUI thatprovides a color channel histogram representation of an image and allowsthe user to edit the image by manipulating the histogram representationmore than once. Specifically, FIG. 23 illustrates the image-editingoperation at four different stages 2305, 2310, 2315, and 2320. Each ofthese stages will be described in more detail below.

As shown in FIG. 23, the GUI 2300 illustrates an image 2325 and a colorchannel histogram representation 2340 of the color values of the image2325. The first stage 2305 illustrates the GUI 2300 of the media-editingapplication when the image 2325 is opened. As shown, the image 2325 andthe color channel histogram representation 2340 of the color values ofthe image 2325 are displayed in the GUI 2300. The image 2325 is apicture of a person canoeing under a sunny sky. The image 2325 is darkdue to underexposure, as illustrated by the diagonal lines across theimage 2325. The image 2325 also has very low contrast. As such, thecolor channel histogram shows a high concentration of color values inthe middle of the histogram.

The second stage 2310 illustrates the GUI 2300 after the user hasmanipulated the histogram representation by providing a first set ofinputs on the color channel histogram representation 2340. In thisexample, the user has selected a location on the histogram 2340 and hasmoved it to the right, as indicated by the arrow 2345. As a result ofthis first set of inputs, the area in the image 2325 near the sun (i.e.,the area of the image 2325 with relatively high luminance values) havebrightened up, as indicated by the removal of the diagonal lines aroundthe area.

The third stage 2315 illustrates the GUI 2300 after the user has begunto provide a second manipulation on the histogram representation 2340.As shown, the user has selected a second location 2350 on the histogramrepresentation 2340 and began to move the cursor to the left, asindicated by the arrow 2355. In the fourth stage 2320, the user hasmoved the cursor almost to the left edge of the histogram representation2340. As a result of the second manipulation, the area of the image 2325that displays the canoe and the sea (i.e., the area of the image 2325with relatively low luminance values) have been darkened.

B. Color Pots

In addition to directly manipulating a color channel histogram of thecolor values of an image, some embodiments also provide a colorselection UI element on the color channel histogram of the color valuesof the image. The color selection UI element allows a user to select aspecific color for editing the set of images. Different embodimentsimplement the color selection UI element differently. In someembodiments, the color selection UI element is a color pot that displaysdifferent colors within the color space. The user can select a specificcolor by selecting a location that represents the specific color withinthe color space that is displayed within the color pot. In otherembodiments, the color selection UI element includes input fields for auser to enter a specific set of color values.

The color selection UI element may be activated in different manners. Insome embodiments, the application provides a set of selectable UI itemson the histogram for activating the color selection interface. The setof selectable UI items are located at different vertical locations onthe histogram. Each vertical location on the histogram corresponds to adifferent luminance value. In these embodiments, the user activates thecolor selection UI element by selecting one of the selectable UI itemson the histogram. In other embodiments, the application provides a hotkey for activating the color selection UI element. In these embodiments,the application activates the color selection UI element based on thehot key and the location of a user's input (e.g., location of a cursor,location of a touch on a touch screen device, etc.) on the histogram.The application then corresponds a particular luminance value thatcorresponds to the location of the user's input. Based on the identifiedluminance value and the selected color, the media-editing application ofsome embodiments generates a color space transform that can be appliedto a set of images.

In some embodiments, the color space transform changes each color withinthe color space to be more similar to the selected color. In someembodiments, the changes that are applied to the different colors arenot constant. For example, larger adjustments are applied to colorshaving luminance values closer to the identified luminance value, andsmaller adjustments are applied to colors having luminance valuesfarther away from the identified luminance value.

FIG. 24 illustrates an example GUI that provides a color pot on a colorchannel histogram representation of the color values of an image andallows a user to edit the image by selecting a color from the color pot.Specifically, FIG. 24 illustrates the image-editing operation at fourdifferent stages 2405, 2410, 2415, and 2420. Each of these stages willbe described in more detail below.

As shown in FIG. 24, the GUI 2400 includes an image 2425 and a colorchannel histogram representation 2440 of the color values of the image2425. The first stage 2405 illustrates the GUI 2400 of the media-editingapplication when the image 2425 is opened. The image 2425 is a pictureof a person walking under a bright dome light. The color temperature ofthe picture 2425 appears to be too cool (i.e., the image has a bluetint), as indicated by the diagonal lines across the image 2425. Thiscan be shown by the dome light appeared to be slightly bluish, ratherthan white. The color channel histogram 2440 includes several selectableUI items 2445, 2450, and 2455 placed at different locations of thehistogram 2440. In this example, a user may activate a color pot byselecting one of the selectable UI items 2445, 2450, and 2455.

The second stage 2410 illustrates the GUI 2400 after the user hasactivated a color pot on the histogram. In this example, the useractivates the color pot 2460 by selecting the selectable UI item 2455.As shown, after the user has selected the UI item 2455, a color pot 2460appears on top of the selectable UI item 2455. The color pot 2460displays different selectable colors for editing the image 2425.

The third stage 2415 illustrates the GUI 2400 after the user has made aselection of a particular color within the color pot 2460. As shown, theuser has selected a yellow color by placing a cursor on top of theyellow region on the color pot 2460. The media-editing application thengenerates a color space transform based on the selection of the yellowcolor and the luminance value that corresponds to the location of theselectable UI item 2455. At the third stage 2415, the application alsoapplies the color space transform to the image 2425 by replacing eachcolor value in the image to an adjusted color value based on the colorspace transform. As a result of the selection of the yellow color, theblue tint is removed from the image 2425 (as indicated by thedisappearing of the diagonal lines on the image 2425). As shown, thedome light in the image 2425 appears to be white, rather than blue.

The fourth stage 2420 illustrates the GUI 2400 after the user hasdeactivated the color pot. In some embodiments, the color pot 2460 isautomatically deactivated after the user has selected a color from thecolor pot. As shown, the color pot 2460 has disappeared (deselected)from the histogram representation 2440 after the user has finishedselecting a color in the third stage 2415.

FIG. 25 conceptually illustrates a process 2500 that is performed by amedia-editing application of some embodiments for editing an image basedon the color selection on the color pot. The process begins by receiving(at 2505) a color pot initialization event on a color channel histogramrepresentation of the color values of a particular image. In someembodiments, the color pot initialization event is a selection of aselectable UI items on the histogram representation.

Once the color pot initialization event is received, the processdisplays (at 2510) a color pot on the histogram representation. In someembodiments, the color pot displays different colors in the color space.Next, the process receives (at 2515) a selection of a color from thecolor pot. In some embodiment, the selected color corresponds to a setof chrominance values.

The process then determines (at 2520) a transform to a color space basedon the selected color. As mentioned above, a color space transform mapseach set of color values in the color space to another set of colorvalues. In some embodiments, the color space transform causes at leastsome of the colors within the color space to be closer (i.e., moresimilar) to the selected color.

Next, the process applies (at 2525) the color space transform to a setof images that includes the particular image. In some embodiments, theset of images are picture frames of a video clip. In other embodiments,the set of images only includes the particular image.

After applying the color space transform to the set of images, theprocess updates (at 2530) the histogram representation according to theadjustments to the particular image. As mentioned above, differentembodiments generate different color space transforms based on the sameset of user inputs. Depending on the color space transforms that wasgenerated, the histogram representation may be updated in differentmanners. The process then determines (at 2535) if there is another colorselection. If there is another color selection, the process returns to2520 to receive another color selection, and will cycle throughoperations 2520-2535 until there is no more color selection provided onthe color pot.

If there are no more color selections, the process determines (at 2540)if the color pot is deselected. In some embodiments, the color pot maybe deselected by making a selection outside of the color pot. If thecolor pot is not deselected, the process returns to 2535 to determine ifthere is another color selection, and will cycle through operations2535-2540 until the color pot is deselected. If the color pot isdeselected, the process removes (at 2545) the color pot from display.Then the process ends.

C. Histogram Representation with Broadened Range

Some embodiments of the invention provide a color channel histogram foran image that includes a range beyond the predetermined black and whiteluminance cutoff value values of the image. The black luminance cutoffvalue identifies a luminance value as black and the white luminancecutoff value identifies a luminance value as white when the image isdisplayed or reproduced.

Pixels of an image may contain luminance values that are beyond therange of black and white luminance cutoff value values under differentscenarios. For example, modern image-capturing devices may store colorvalues beyond the black and white luminance cutoff value values in animage. Also, luminance values of some pixels that are originally withinthe range of the black and white luminance cutoff value values may bepushed beyond the range by a user editing the image (e.g., increasing ordecreasing the brightness level of at least some pixels of the image).Thus, the media-editing application of some embodiments provides a colorchannel histogram representation that includes a range beyond thepredetermined black and white luminance cutoff value values. That way,all pixels of the image, including pixels with luminance values darkerthan the black luminance cutoff value and pixels with luminance valuesbrighter than the white luminance cutoff value, will be accuratelypresented in the histogram.

FIG. 26 illustrates an example GUI that provides a color channelhistogram representation with broadened luminance range for an image. Asshown in the FIG. 26, the GUI 2600 includes an image 2625 and a colorchannel histogram representation 2640 of the color values of the image2625. The histogram representation 2640 in this example shows adistribution of luminance values across the image 2625. The GUI 2600also includes a check box for activating the display of high dynamicdata of the image 2625. The image 2625 is a picture of a room thatincludes a window on the left and a painting on the right. The leftportion of the image 2625 is extremely bright while the right portion ofthe image 2625 is extremely dark. As such, the luminance values of atleast some pixels around the area of the window (i.e., displayed aswhite) are brighter than the white luminance cutoff value of the image.In addition, the luminance values of at least some pixels around thearea near the painting (i.e., displayed as black) are darker than theblack luminance cutoff value.

The first stage 2605 illustrates the GUI 2600 when the application hasjust opened the image file 2625. As shown, since the check box foractivating the display of high dynamic data has not been selected yet,the histogram representation 2640 only includes a luminance range from0-1 (i.e., the luminance range between the black and white luminancecutoff value values). In addition, the representation of the luminancevalues that are beyond the black and white luminance cutoff value valuesare packed at the two ends of the histogram 2640, as indicated by thespikes at the two ends of the graph 2640.

The second stage 2610 is a transition stage. At this stage, the useractivates the high dynamic range histogram representation (e.g., byselecting a selectable item 2645 for activating the high dynamic rangehistogram representation). The third stage 2615 illustrates the GUI 2600after the high dynamic range histogram representation is activated. Asshown, the color channel histogram representation has extended the rangebeyond the black and white cutoff value values (indicated by the labels0 and 1). In addition, luminance values of the image that are darkerthan the black cutoff value and brighter than the white cutoff value inthe image 2625 are accurately represented in the color channel histogramrepresentation 2640 in this stage.

By accurately presenting pixel data in a color channel histogramrepresentation, a user may properly analyze and edit the image to adesired effect. For example, after learning that some of the pixels haveluminance values that are brighter than the white luminance cutoff value(displayed as pure white) and darker than the black luminance cutoffvalue (displayed as pure black), a user may edit the image by bringingthe luminance values of those pixels back within the range of the blackand white luminance cutoff value values. In some embodiments, the usermay perform such an edit by manipulating directly on the color channelhistogram representation of the color values of the image.

The example above illustrates a histogram representation that contains abroadened range of luminance values. However, as any color channel thatrepresents the color values of the image has a minimum color value and amaximum color value (e.g., for a color channel that uses 8 bits torepresent its color, the minimum value may be 0 and the maximum valuemay be 255), color clipping may occur not only in the luminance channel,but also in any color channels (e.g., blue channel, red channel, greenchannel, etc.). Thus, the application of some embodiments may alsodisplay other types of histogram representation that contains abroadened range of other color channels' values.

FIG. 27 illustrates an example GUI of such an image-editing operation.Specifically, FIG. 27 illustrates the image-editing operation at fourdifferent stages 2705, 2710, 2715, and 2720. Each of these stages willbe described in more detail below.

As shown in FIG. 27, the GUI 2700 includes an image 2725 and a colorchannel histogram representation 2740 of the color values of the image2725. The first stage 2705 illustrates the GUI 2700 of the media-editingapplication when the image file 2725 is opened. As shown, the image 2725is a picture of a room with a window overlooking a garden. Because thegarden outside of the room is much brighter than the room, most of thepixels around the area of the window in the image 2725 are displayed aswhite. The color channel histogram representation 2740 includes a rangebeyond the black and white cutoff value values of the image. As such,even though the pixels that are brighter than the white luminance cutoffvalue are displayed as white in the image 2725, they are accuratelypresented in the color channel histogram 2740. Specifically, the portionof the histogram 2745 represents pixels with luminance values higherthan the white cutoff value.

The second stage 2710 illustrates the GUI 2700 after the user hasinitiated the image editing operation by selecting a location on thecolor channel histogram representation 2740. The selection of thelocation on the histogram 2740 may be performed by placing a cursor at adesired location on the histogram 2740 and providing an input (e.g.,clicking on the cursor controlling device, pressing a hot key, etc. ortouching an area on a touch screen device that displays the histogram2740). As shown, the user has selected a location 2750 on the histogram2740 by placing a cursor 2755 over the location 2750. In this example,the selected location 2750 corresponds to a luminance value that isbrighter than the white cutoff value.

The third stage 2715 illustrates the GUI 2700 after the user has begunmanipulating the histogram representation 2740 by providing a vector onthe histogram representation 2740. In some embodiments, the user canprovide a vector by moving the cursor (or the finger on a touch screendevice) to a different location. As shown, the user has started to movethe cursor 2755 to the left, as indicated by the arrow 2760. Based onthe selected location 2750 on the histogram 2740 and the movement of thecursor 2755, the application of some embodiments generates a color spacetransform that is applied to the image 2725. Specifically, the colortransform reduces the luminance values of the color values havingluminance values above the white cutoff value.

As shown in the stage 2715, the application edits the image 2725according to the color space transform that is generated based on theuser inputs. In particular, more details of the garden outside of thewindow are shown in the image 2725 in the third stage 2715 when comparedto the image 2725 of the second stage 2710.

The fourth stage 2720 illustrates the GUI 2700 after the user hascompleted the image editing operation. As shown, the user has moved thecursor 2755 further to the left. In some embodiments, the applicationgenerates a new color space transform based on the new movement of thecursor 2755. This new color space transform further reduces theluminance values of some of the pixels that have high luminance values.As shown, more details of the garden has appeared in the image 2740 as aresult of the cursor movement.

Pixels may contain luminance values beyond the black and white luminancecutoff value values for many reasons. For example, modern image/videocapturing devices may sometimes store luminance values that are beyondthe black and white cutoff value values range when the image or video iscaptured. Images may also contain luminance values beyond the black andwhite luminance cutoff value values range as a result of a user'sediting of the images. FIG. 28 illustrates an example GUI for bringingthe luminance values of an image back within the black and white cutoffvalue range after a user pushes some of the luminance values beyond therange.

Specifically, FIG. 28 illustrates the image-editing operation at fourdifferent stages 2805, 2810, 2815, and 2820. Each of these stages willbe described in more detail below.

As shown in FIG. 28, the GUI 2800 includes an image 2825, a colorchannel histogram representation 2840 of the color values of the image2825, and color adjusting sliders 2845 and 2850. The different coloradjusting sliders are for adjusting the color values of the image 2825.For example, the slider 2845 is for adjusting brightness of the image2825 and the slider 2850 is for adjusting contrast of the image 2825.

The image editing operation will now be described by reference to thestate of the GUI 2800 during the four stages 2805, 2810, 2815, and 2820.The first stage 2805 illustrates the GUI 2800 of the media-editingapplication when the image 2825 is opened. As shown, the image 2825 isan image of a room with a window on the left and a painting on theright. The image 2825 also shows two flowerpots sitting outside thewindow. In addition, the right side of the painting appears to be toodark, as indicated by the diagonal lines across that portion of theimage 2825. The color channel histogram representation 2840 indicatesthat all color values of the image 2825 are within the range of theblack and white luminance cutoff value values.

The second stage 2810 illustrates the GUI 2800 after the user hasbrightened the entire image 2825 by sliding the brightness slider 2845to the right. As a result of the brightness adjustment, the shadowaround the painting has disappeared. In addition, the two flowerpotsoutside the window have been blown out (i.e., displayed as white) as theluminance values of the pixels that represent the flowerpots are nowhigher than the white luminance cutoff value. The color channelhistogram representation 2840 also shows that the color values of aparticular part of the image 2825 have been pushed beyond the whiteluminance cutoff value (labeled as 1 in the graph).

The third stage 2815 illustrates the GUI 2800 after the user has begunto bring some of the color values of the image 2825 back within therange of the black and white luminance cutoff values by manipulating thehistogram representation 2840. As shown, the user has selected alocation 2855 on the histogram representation 2840 and began to drag thecursor to the left.

The fourth stage 2820 illustrates the GUI 2800 after the user hascompleted the image editing operation. As shown, the user has moved thecursor further to the left. As a result of the user's input on thehistogram representation 2840, the luminance values of the color valuesthat are beyond the white luminance cutoff value have been bought backwithin range and the flower pots have reappeared in the image 2825.

IV. On-Screen Control

In addition to allowing a user to edit an image by directly manipulatingthe graphical representations of the image, some embodiments display aset of color adjustment tools on the image and allow a user to edit thecolor values of the image by controlling the color adjustment tools thatare displayed on the image. In some embodiments, the color adjustmenttools include a value adjustment tool (e.g., a range slider) foradjusting pixel values of different color channels. In otherembodiments, the application displays a color adjustment tool thatsimulates a graphical representation of the color values of the image.In these embodiments, the user may edit the image by providing anadditional input (e.g., a vector) on the displayed image as if the useris manipulating the actual graphical representation of the image.

FIG. 29 illustrates an example GUI that displays a color adjustment toolon an image and allows a user to edit the image by controlling the coloradjustment tool. Specifically, FIG. 29 illustrates the image-editingoperation at four different stages 2905, 2910, 2915, and 2920. Each ofthese stages will be described in more detail below.

As shown in FIG. 29, the GUI 2900 illustrates an image 2925 and avectorscope representation 2935 of the color values of the image 2925.The first stage 2905 illustrates the GUI 2900 of the media-editingapplication before the user activates the color adjustment tools on thedisplayed image 2925. As shown, the image 2925 is a picture of a red andwhite house and the front lawn of the house. As the dominant colors inthe image 2925 are red, green, and blue, the vectorscope representation2935 shows a high concentration of colors in the red, green, and bluesections.

The second stage 2910 illustrates the GUI 2900 after the user hasinitiated an image editing operation by activating the color adjustmenttool on the displayed image 2925. The user may activate the coloradjustment tools in different manners. In some embodiments, the user mayactivate the color adjustment tools by selecting a location on theimage. In these embodiments, a user can select a location on the graphby placing a cursor on the image and providing an input (e.g., clickinga button of a cursor controlling device, pressing a hot key, etc. or bytouching an area of a touch screen display that displays the graph). Themedia-editing application then determines a position of the user input(the cursor position or the position of the touch) and corresponds thatposition to a particular pixel on the image. The position of the userinput can be determined by identifying a set of coordinates (e.g., ahorizontal coordinate x and a vertical coordinate y) of the user input.

As shown, the user has selected a location 2950 on the image 2925 byplacing a cursor at the location 2950. Based on the selected location onthe image, the application of some embodiments identifies a particularpixel on the image. In some embodiments, the application uses the colorvalues of the particular pixel to determine a color space transform forthe image.

As a result of the selection, the application displays a set of coloradjustment tools on the image 2925. Different embodiments providedifferent color adjustment tools (2945 and 2970-2985) for display on theimage. In this example, the set of color adjustment tools include fourrange sliders 2970-2985 for adjusting color values of the image 2925 anda color adjustment tool 2945 that simulates a vectorscope representationof the color values of the image. The user may provide an additionalinput (e.g., a vector) on the color adjustment tool 2945 as if the useris manipulating the actual vectorscope representation.

As shown, the set of color adjustment tools are displayed around theselected area on the image 2925. Additionally, the applicationcorresponds the color values of the particular pixel to a point on thevectorscope representation 2935, as indicated by a white circle 2960 onthe vectorscope 2935. Although the set of color adjustment tools isshown to be opaque in this example, some embodiments display a set oftranslucent color adjustment tools on the image in order not to obstructthe view of the image from the user.

The third stage 2915 illustrates the GUI 2900 after the user hasprovided a vector on the color adjustment tool 2945. As shown, the userhas started to drag the cursor 2955 toward the upper left corner of theimage 2925, as indicated by the arrow 2990. As shown in the stage 2915,the movement of the cursor 2955 has created a vector 2992 that isapplied to the actual vectorscope representation 2935. The applicationthen generates a color space transform that will be applied to the imagebased on user inputs. Specifically, the media-editing application usesthe color values of the particular pixel and the vector input todetermine a color space transform in the same manner as if the inputsare provided directly on the actual vectorscope representation. Theapplication then edits the image and updates the vectorscoperepresentation of the color values of the image according to the colorspace transform. As a result, the pixels that are green in color (e.g.,pixels around the lawn and the trees) have been slightly changed toorange.

The fourth stage 2920 illustrates the GUI 2900 after the user hascompleted the media editing operation. As shown, the user has moved thecursor 2955 further to the upper left corner of the vectorscope (asindicated by the arrow 2994) and has stopped. This movement provides anew vector 2996 on the displayed image 2925. Similar to the third stage2915, the application creates a new color space transform based on thenew inputs and applies the transform to the image 2925. As a result, thepixels that were originally green (e.g., pixels of the trees and lawn)have become more orange than they were in the third stage 2915, asindicated by the cross hatching on the grass area of the image.

In the above example, the user has edited the image 2925 by manipulatingthe color adjustment tool 2945 on the displayed image. The user may alsoedit the image 2925 by adjusting one or more of the range sliders2970-2985. Each slider is for adjusting different aspects of the colorvalues. For example, the slider 2970 is for adjusting luminance valuesof the image 2925, the slider 2975 is for adjusting red color values ofthe image 2925, the slider 2980 is for adjusting blue color values ofthe image 2925, and the slider 2985 is for adjusting green color valuesof the image 2925. When the user adjusts a range slider, the applicationalso generates a color space transform based on the color values of theparticular pixel selected by the user and the adjustment to the rangeslider. For example, when a user adjusts the luminance channel rangeslider, the application may generate the color space transform thatapplies different luminance adjustments to the pixels based on the colorvalues of the pixels. That is, larger adjustments are applied to pixelswith color values that are more similar to the color values of theparticular pixel and smaller adjustments are applied to pixels withcolor values that are further away from the color values of theparticular pixel.

In addition to providing a color adjustment tool that simulates avectorscope representation, the media-editing application of someembodiments may provide color adjustment tools that simulate othergraphical representations (e.g., waveform, histogram, etc.) of the colorvalues of the image. In these embodiments, a user may provide additionalinput (e.g., a hotkey, etc.) to select a color adjustment tool thatsimulates a particular graphical representation. FIG. 30 illustrates anexample GUI that provides a color adjustment tool on an image thatsimulates a histogram representation of the color values of the image.Specifically, FIG. 30 illustrates the image-editing operation at fourdifferent stages 3005, 3010, 3015, and 3020. Each of these stages willbe described in more detail below.

As shown in FIG. 30, the GUI 3000 illustrates an image 3025 and ahistogram representation 3035 of the color values of the image 3025. Inthis example, the histogram representation 3035 shows a distribution ofluminance values across the image. The first stage 3005 illustrates theGUI 3000 of the media-editing application before the user activates thecolor adjustment tools on the displayed image 3025. As shown, the image3025 is a picture of a red and white house and the front lawn of thehouse. As the majority of the image 3025 has modest brightness levels,with the white roof being extremely bright, the histogram representation3035 indicates a high concentration of pixels in the middle of the graphwith a spike at the right end of the graph that represents pixels of thewhite roof.

The second stage 3010 illustrates the GUI 3000 after the user hasinitiated an image editing operation by activating the color adjustmenttool on the displayed image 3025. As shown, the user selects a location3050 on the image 3025 by placing a cursor 3055 on top of an areashowing the tree on the image 3025. Based on the selection of thelocation 3050 on the image, the application of some embodimentidentifies a particular pixel on the image. In some embodiments, theapplication uses the color values of the particular pixel to determine acolor space transform for the image.

As a result of the selection, the application displays a coloradjustment tool 3045 on the image 3025. Different embodiments providedifferent color adjustment tools for display on the image. In thisexample, the color adjustment tool 3045 simulates a histogramrepresentation of the color values of the image 3025. The user mayprovide an additional input (e.g., a vector) on the displayed image asif the user is manipulating the actual vectorscope representation.

As shown, the color adjustment tool 3045 is displayed around theselected area on the image 3025. Additionally, the applicationcorresponds the luminance value of the particular pixel to a point onthe histogram representation 3035, as indicated by a white circle 3060on the histogram 3035.

The third stage 3015 illustrates the GUI 3000 after the user hasprovided a vector input on the displayed image 3025. As shown, the userhas started to drag the cursor 3055 to the left of the image, asindicated by the arrow 3070. The movement of the cursor 3055 has createda vector 3060 that is applied to the actual histogram representation3035. The application then generates a color space transform that isapplied to the image based on user inputs. Specifically, themedia-editing application uses the color values of the particular pixeland the vector input to determine a color space transform in the samemanner as if the inputs are provided directly on the actual histogramrepresentation. The application then edits the image and updates thehistogram representation of the color values of the image according tothe color space transform. As a result, the pixels that have relativelylow luminance values (e.g., pixels of the trees and shadows) have beenslightly darkened.

The fourth stage 3020 illustrates the GUI 3000 after the user hascompleted the media editing operation. As shown, the user has moved thecursor 3055 further to the left (as indicated by the arrow 3075) and hasstopped at a new location. This latest movement of the cursor provides anew vector 3065 that is applied to the histogram representation 3035.Similar to the third stage 3015, the application creates a new colorspace transform based on the new inputs and applies the transform to theimage 3025. As a result, the pixels that have relatively low luminancevalues are further darkened.

Different embodiments use different techniques to allow a user to selecta location on the image and activate the color adjustment tools. FIG. 31illustrates one example approach. Under this approach, a user can selecta point on the image by placing a cursor on the image and providing aninput (e.g., clicking a button of a cursor controlling device, pressinga hot key, etc. or by touching a touch screen display at an area thatdisplays the image). The media-editing application of some embodimentsdetermines a position of the user input (the cursor position or thelocation of the touch) and corresponds that position to a pixel on theimage. In some embodiments, the position of the user input can bedetermined by identifying a set of coordinates (e.g., a horizontalcoordinate x and a vertical coordinate y) of the user input.

FIG. 32 conceptually illustrates a process 3200 that is performed by amedia-editing application of some embodiments for editing an image basedon user inputs provided directly on a displayed image. The processbegins by displaying (at 3205) a particular image on the GUI of themedia-editing application.

Next, the process receives (at 3210) an activation event for theon-screen control. In some embodiments, the activation event is aselection of a location on the displayed image. As mentioned above, theselection can be made by placing a cursor on the image and providing aninput (e.g., clicking a button of a cursor controlling device, pressinga hot key, etc. or by touching a touch screen display at an area thatdisplays the image). In some embodiments, the process identifies aparticular pixel on the image based on the selected point.

After the process has received an activation event for the on-screencontrol, the process displays (at 3215) a set of color adjustment toolson the displayed image for adjusting different color values of theimage. Different embodiments display different types of color adjustmenttools on the image. In some embodiments, the set of color adjustmenttools include one or more range sliders for adjusting a particular colorchannel (e.g., adjusting the luminance values, the red color values,etc.). Instead of or in addition to the range sliders, the coloradjustment tool may simulate a graphical representation of the colorvalues of the image (e.g., vectorscope, waveform, histogram, etc.). Inthese embodiments, the user may edit the image by providing anadditional input (e.g., a vector) on the image as if the user ismanipulating the actual graphical representation in a manner similar tothat as described above in Sections I-III.

Then, the process receives (at 3220) an input on a color adjustmenttool. The input may be a movement of the slider bar or a vector.

Next, the process determines (at 3225) a color space transformed basedon the user inputs. In some embodiments, the color space transformed isdetermined based on the color values of the particular pixel and thereceived input on the color adjustment tool.

The process then applies (at 3230) the color space transform to a set ofimage that includes the particular image. In some embodiments, the setof images are picture frames of a video clip. In other embodiments, theset of images only include the particular image.

After the process has applied the color space transform to the set ofimages, the process determines (3235) if another input is received onthe displayed color adjustment tools. If another input is received onthe color adjustment tools, the process returns to 3225 to determine acolor space transform based on the newly received input, and will cyclethrough operations 3225-3235 until no more inputs are received on thecolor adjustment tools. If no more inputs are received on the coloradjustment tools, the process determines (at 3240) if a deactivationevent for the on-screen control is received. In some embodiments, theon-screen control may be deactivated by pressing a hotkey or a selectionoutside of the displayed image. If the on-screen control is notdeactivated, the process returns to 3235 to determine if there isanother input on the displayed color adjustment tools, and will cyclethrough operations 3235-3240 until the on-screen control is deactivated.If the on-screen control is deactivated, the process removes (at 3245)the color adjustment tools from the displayed image. Then the processends.

V. Software Architecture

In some embodiments, the processes described above are implemented assoftware running on a particular machine, such as a computer or ahandheld device, or stored in a machine readable medium. FIG. 33conceptually illustrates the software architecture of a media-editingapplication 3300 of some embodiments. Some examples of suchmedia-editing application include iPhoto®, iMovie® and Final Cut Pro®,all sold by Apple Inc.®

In some embodiments, the media-editing application is a stand-aloneapplication or is integrated into another application, while in otherembodiments the application might be implemented within an operatingsystem. Furthermore, in some embodiments, the application is provided aspart of a server-based solution. In some such embodiments, theapplication is provided via a thin client. That is, the application runson a server while a user interacts with the application via a separatemachine remote from the server.

In other such embodiments, the application is provided via a thickclient. That is, the application is distributed from the server to theclient machine and runs on the client machine.

As shown, the media-editing application 3300 includes a user interface(UI) interaction module 3305, a video rendering module 3310, a set ofvideo editing modules 3315, a media import module 3320, a color graphsgenerator 3325, a color space transform generator 3330, and a coloradjustment engine 3335. The application also includes a media storage3340. In some embodiments, the media storage 3340 is a set of filefolders organized by the media-editing application and stored on aparticular set of storage devices. The storage devices may include theboot drive of the electronic device on which the application operates, adifferent partition of that disk, a separate internal or external harddrive, a flash drive, SD card, etc.

The UI interaction module 3305 of the media-editing application 3300interprets the user input data received from the input device drivers3345 and passes it to various modules, including the video editingmodules 515, the media import module 520, and the color space transformgenerator 3330. In some embodiments, the input data directly affects thecomposite presentation data or other data stored in the media storage3340.

The UI interaction module 3305 also manages the display of the UI, andoutputs this display information to the display drivers 3350. This UIdisplay information may be based on information from the variousmodules, including the video editing modules 3315, the video renderingmodule 3310, the media import module 3320, the color graphs generator3325, and the color adjustment engine 3335.

The media import module 3320 imports media (e.g., an image, a videocontaining multiple picture frames, etc.) into the media-editingapplication for use. Some embodiments, as shown, receive the mediadirectly from a video capturing device such as a video camera 3355. Someembodiments import media from an external storage 3360. The externalstorage 3360 may be an SD card, a flash drive, an external hard drive,an internal hard drive in which the files are not stored in theorganized file folder structure of the application, etc.

The color graphs generator 3325 generates different graphicalrepresentations for an image. In some embodiments, the differentgraphical representations are generated based on the color values of theimage. Some example graphical representations include waveformrepresentation (as shown in Section I), vectorscope representation (asshown in Section II), and color channel histogram representation (asshown in Section III). In some embodiments, the graphicalrepresentations generated by the color graphs generator 3325 areprovided to the UI interaction module 3305 for display on a displaydevice.

The color space transform generator 3330 takes the inputs received fromthe UI interaction module 3305 and generates a transform to a colorspace. In some embodiments, the inputs are associated with directlymanipulating a graphical representation, as described above in SectionsI-III. In some embodiments, the color space transform that maps colorsdefined within a color space in which the color values of an image aredefined to a different set of colors.

The color adjustment engine 3335 applies the color space transformgenerated by the color space transform generator 3330 to a set ofimages. In some embodiments, the set of images are picture frames of avideo clip.

FIG. 33 also illustrates an operating system that includes input devicedriver(s) 3345 and display drivers 3350. In some embodiments, asillustrated, the device drivers 3345 and display drivers 3350 are partof the operating system even when the media editing application 3300 isan application separate from the operating system.

The input device drivers 3345 may include drivers for translatingsignals from a keyboard, mouse, touchpad, drawing tablet, touchscreen,etc. A user interacts with one or more of these input devices, whichsend signals to their corresponding device driver. The device driverthen translates the signals into user input data that is provided to theUI interface interaction module 3305.

The present application describes a graphical user interface thatprovides users with numerous ways to perform different sets ofoperations and functionalities. In some embodiments, these operationsand functionalities are performed based on different commands that arereceived from users through different input devices (e.g., keyboard,trackpad, touchpad, mouse, etc.). For example, the present applicationillustrates the use of a cursor in the graphical user interface tocontrol (e.g., select, move) objects in the graphical user interface.However, in some embodiments, objects in the graphical user interfacecan also be controlled or manipulated through other controls, such astouch control. In some embodiments, touch control is implemented throughan input device that can detect the presence and location of touch on adisplay of the input device. An example of a device with suchfunctionality is a touch screen device (e.g., as incorporated into asmart phone, a tablet computer, etc.). In some embodiments with touchcontrol, a user directly manipulates objects by interacting with thegraphical user interface that is displayed on the display of the touchscreen device. For instance, a user can select a particular object inthe graphical user interface by simply touching that particular objecton the display of the touch screen device. As such, when touch controlis utilized, a cursor may not even be provided for enabling selection ofan object of a graphical user interface in some embodiments. However,when a cursor is provided in a graphical user interface, touch controlcan be used to control the cursor in some embodiments.

VI. Electronic System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or morecomputational or processing unit(s) (e.g., one or more processors, coresof processors, or other processing units), they cause the processingunit(s) to perform the actions indicated in the instructions. Examplesof computer readable media include, but are not limited to, CD-ROMs,flash drives, random access memory (RAM) chips, hard drives, erasableprogrammable read only memories (EPROMs), electrically erasableprogrammable read-only memories (EEPROMs), etc. The computer readablemedia does not include carrier waves and electronic signals passingwirelessly or over wired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storagewhich can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

FIG. 34 conceptually illustrates an electronic system 3400 with whichsome embodiments of the invention are implemented. The electronic system3400 may be a computer (e.g., a desktop computer, personal computer,tablet computer, etc.), phone, PDA, or any other sort of electronicdevice. Such an electronic system includes various types of computerreadable media and interfaces for various other types of computerreadable media. Electronic system 3400 includes a bus 3405, processingunit(s) 3410, a graphics processing unit (GPU) 3415, a system memory3420, a network 3425, a read-only memory 3430, a permanent storagedevice 3435, input devices 3440, and output devices 3445.

The bus 3405 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 3400. For instance, the bus 3405 communicativelyconnects the processing unit(s) 3410 with the read-only memory 3430, theGPU 3415, the system memory 3420, and the permanent storage device 3435.

From these various memory units, the processing unit(s) 3410 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Someinstructions are passed to and executed by the GPU 3415. The GPU 3415can offload various computations or complement the image processingprovided by the processing unit(s) 3410. In some embodiments, suchfunctionality can be provided using CoreImage's kernel shading language.

The read-only-memory (ROM) 3430 stores static data and instructions thatare needed by the processing unit(s) 3410 and other modules of theelectronic system. The permanent storage device 3435, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system3400 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 3435.

Other embodiments use a removable storage device (such as a floppy disk,flash memory device, etc., and its corresponding disk drive) as thepermanent storage device. Like the permanent storage device 3435, thesystem memory 3420 is a read-and-write memory device. However, unlikestorage device 3435, the system memory 3420 is a volatile read-and-writememory, such a random access memory. The system memory 3420 stores someof the instructions and data that the processor needs at runtime. Insome embodiments, the invention's processes are stored in the systemmemory 3420, the permanent storage device 3435, and/or the read-onlymemory 3430. For example, the various memory units include instructionsfor processing multimedia clips in accordance with some embodiments.From these various memory units, the processing unit(s) 3410 retrievesinstructions to execute and data to process in order to execute theprocesses of some embodiments.

The bus 3405 also connects to the input and output devices 3440 and3445. The input devices 3440 enable the user to communicate informationand select commands to the electronic system. The input devices 3440include alphanumeric keyboards and pointing devices (also called “cursorcontrol devices”), cameras (e.g., webcams), microphones or similardevices for receiving voice commands, etc. The output devices 3445display images generated by the electronic system or otherwise outputdata. The output devices 3445 include printers and display devices, suchas cathode ray tubes (CRT) or liquid crystal displays (LCD), as well asspeakers or similar audio output devices. Some embodiments includedevices such as a touchscreen that function as both input and outputdevices.

Finally, as shown in FIG. 34, bus 3405 also couples electronic system3400 to a network 3425 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or an Intranet,or a network of networks, such as the Internet. Any or all components ofelectronic system 3400 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself. In addition, someembodiments execute software stored in programmable logic devices(PLDs), ROM, or RAM devices.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium,” “computer readable media,” and “machinereadable medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by a computer. Theseterms exclude any wireless signals, wired download signals, and anyother ephemeral signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. In addition, a number of the figures(including FIGS. 2, 10, 15, 16, 21, 25, and 32) conceptually illustrateprocesses. The specific operations of these processes may not beperformed in the exact order shown and described. The specificoperations may not be performed in one continuous series of operations,and different specific operations may be performed in differentembodiments. Furthermore, the process could be implemented using severalsub-processes, or as part of a larger macro process. Thus, one ofordinary skill in the art would understand that the invention is not tobe limited by the foregoing illustrative details, but rather is to bedefined by the appended claims.

What is claimed is:
 1. A non-transitory machine readable medium having acomputer program for adjusting color values of an image represented in acolor space, the computer program comprising sets of instructions forexecution by at least one processing unit, the sets of instructionscomprising: displaying a graphical representation of the color values ofthe image; receiving a set of inputs on the graphical representation; inresponse to the received set of inputs, determining a color spacetransform for mapping a set of color values defined within the colorspace to a different set of color values; and modifying the image byusing the color space transform to change the color values of the image.2. The machine readable medium of 1, wherein the received set of inputscomprises a selection of a location on the graphical representation. 3.The machine readable medium of 2, wherein the received set of inputsfurther comprises a vector.
 4. The machine readable medium of 2, whereinthe selected location corresponds to a set of color values.
 5. Themachine readable medium of 4, wherein the set of color values comprisesluminance values.
 6. The machine readable medium of 4, wherein the setof color values comprises chrominance values.
 7. The machine readablemedium of 1, wherein the color space transform is implemented as analgorithm that generates an adjusted color value for each color valuewithin the color space, wherein the set of instructions for modifyingthe image comprises a set of instructions for using the algorithm tochange the color values of the image in real-time.
 8. The machinereadable medium of 1, wherein the set of instructions for determiningthe color space transform further comprises a set of instructions forgenerating a look-up table that specifies an adjusted color value foreach color value within the color space.
 9. The machine readable mediumof 8, wherein the set of instructions for modifying the image comprisesa set of instructions for replacing each color value in the image with acorresponding color value based on the generated lookup table.
 10. Themachine readable medium of 1, wherein the computer program furthercomprises a set of instructions for modifying the graphicalrepresentation according to the adjustments made to the color values ofthe image.
 11. The machine readable medium of 1, wherein the image isone of a plurality of picture frames of a video clip, wherein thecomputer program further comprises a set of instructions for modifyingthe plurality of images using the color space transform.
 12. A method ofproviding an application comprising a set of modules for adjusting colorvalues of an image represented in a color space, the method comprising:providing a generation module for generating a graphical representationof the color values of the image; providing a user interface module forreceiving a set of inputs on the graphical representation; providing acolor space transform module for generating a color space transform inresponse to the received set of inputs, the color space transform formapping a set of color values defined within the color space to adifferent set of color values; and providing a color adjustment modulefor modifying the image by using the color space transform to change thecolor values of the image.
 13. The method of 12, wherein the receivedset of inputs comprises a selection of a location on the graphicalrepresentation.
 14. The method of 13, wherein the received set of inputsfurther comprises a vector on the graphical representation having anorigin at the selected location.
 15. The method of 13, wherein theselected location corresponds to a set of color values.
 16. The methodof 15, wherein the set of color values comprises luminance values. 17.The method of 15, wherein the set of color values comprises chrominancevalues.
 18. The method of 12, wherein the color space transform isimplemented as an algorithm that generates an adjusted color value foreach color value within the color space, wherein the color adjustmentmodule modifies the image by using the algorithm to change the colorvalues of the image in real-time.
 19. The method of 12, wherein theimage is one of a plurality of picture frames of a video clip, whereincolor adjustment module is also for modifying the plurality of pictureframes using the color space transform.
 20. A method of providing agraphical user interface (GUI) for editing an image comprising aplurality of pixels in a media editing application, the plurality ofpixels having color values represented by a color space, the methodcomprising: providing a first display area for displaying the image;providing a second display area for displaying a graphicalrepresentation of the color values of the image; and providing a userinterface that allows a user to edit the image by providing a set ofinputs through the displayed graphical representation.
 21. The method of20, wherein a color space transform is generated in response to the setof inputs provided on the displayed graphical representation, whereinthe color space transform maps a set of color values defined within thecolor space to a different set of color values.
 22. The method of 20,wherein the set of inputs comprises a selection of a point on thegraphical representation.
 23. The method of 22, wherein the selectedpoint on the graphical representation corresponds to a set of colorvalues within the color space.
 24. The method of 23, wherein the set ofcolor values comprises a luminance value when the graphicalrepresentation is one of a color channels histogram and a waveform. 25.The method of 23, wherein the set of color values comprises a set ofchrominance values when the graphical representation is a vectorscope.